Skip to contentSkip to conversation

docs/bpf: Add description for CO-RE relocations

PATCH Superseded Eduard Zingerman 3 participants 8 messages Aug 24, 2023 – Aug 25, 2023

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Eduard Zingerman

author maintainer
#0 lore raw

PATCH 1/1 docs/bpf: Add description for CO-RE relocations

Add a section on CO-RE relocations to llvm_relo.rst. Describe relevant .BTF.ext structure, enum bpf_core_relo_kind and struct bpf_core_relo in some detail.
Description is based on doc-string from include/uapi/linux/bpf.h.

---
 Documentation/bpf/btf.rst        |  27 ++++-
 Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
 2 files changed, 201 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +23 -4
index f32db1f44ae9..c0530211c3c1 100644
@@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
 4.2 .BTF.ext section
 --------------------
 
-The .BTF.ext section encodes func_info and line_info which needs loader
-manipulation before loading into the kernel.
+The .BTF.ext section encodes func_info, line_info and CO-RE relocations
+which needs loader manipulation before loading into the kernel.
 
 The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
 and ``tools/lib/bpf/btf.c``.
@@ -745,11 +745,16 @@ The current header of .BTF.ext section::
         __u32   func_info_len;
         __u32   line_info_off;
         __u32   line_info_len;
+
+        /* optional part of .BTF.ext header */
+        __u32   core_relo_off;
+        __u32   core_relo_len;
     };
 
 It is very similar to .BTF section. Instead of type/string section, it
-contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
-about func_info and line_info record format.
+contains func_info, line_info and core_relo sub-sections.
+See :ref:`BPF_Prog_Load` for details about func_info and line_info
+record format.
 
 The func_info is organized as below.::
 
@@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
 bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
 beginning of section (``btf_ext_info_sec->sec_name_off``).
 
+The core_relo is organized as below.::
+
+     core_relo_rec_size
+     btf_ext_info_sec for section #1 /* core_relo for section #1 */
+     btf_ext_info_sec for section #2 /* core_relo for section #2 */
+
+``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
+structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
+within a single ``btf_ext_info_sec`` describe relocations applied to
+section named by ``btf_ext_info_sec::sec_name_off``.
+
+See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
+for more information on CO-RE relocations.
+
 4.2 .BTF_ids section
 --------------------
 
Documentation/bpf/llvm_reloc.rst +178 -0

181 lines not on this page · raw message

Signature · 1 line
2.41.0
reviewer
#1 lore raw
Quoted 4 lines · Eduard Zingerman

Add a section on CO-RE relocations to llvm_relo.rst. Describe relevant .BTF.ext structure, enum bpf_core_relo_kind and struct bpf_core_relo in some detail.
Description is based on doc-string from include/uapi/linux/bpf.h.

Thanks Eduard. This is very helpful to give bpf deverlopers some insight about how different of core relocations are supported in llvm and libbpf.

Some comments below.

Quoted 45 lines
---
  Documentation/bpf/btf.rst        |  27 ++++-
  Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
  2 files changed, 201 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +9 -4
index f32db1f44ae9..c0530211c3c1 100644
@@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
  4.2 .BTF.ext section
  --------------------
 
-The .BTF.ext section encodes func_info and line_info which needs loader
-manipulation before loading into the kernel.
+The .BTF.ext section encodes func_info, line_info and CO-RE relocations
+which needs loader manipulation before loading into the kernel.
 
  The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
  and ``tools/lib/bpf/btf.c``.
@@ -745,11 +745,16 @@ The current header of .BTF.ext section::
          __u32   func_info_len;
          __u32   line_info_off;
          __u32   line_info_len;
+
+        /* optional part of .BTF.ext header */
+        __u32   core_relo_off;
+        __u32   core_relo_len;
      };
 
  It is very similar to .BTF section. Instead of type/string section, it
-contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
-about func_info and line_info record format.
+contains func_info, line_info and core_relo sub-sections.
+See :ref:`BPF_Prog_Load` for details about func_info and line_info
+record format.
 
  The func_info is organized as below.::
 
@@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
  bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
  beginning of section (``btf_ext_info_sec->sec_name_off``).
 

Documentation/bpf/btf.rst kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct · line 804

Quoted 10 lines
Documentation/bpf/btf.rst +10 -0
+The core_relo is organized as below.::
+
+     core_relo_rec_size
+     btf_ext_info_sec for section #1 /* core_relo for section #1 */
+     btf_ext_info_sec for section #2 /* core_relo for section #2 */
+
+``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
+structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
+within a single ``btf_ext_info_sec`` describe relocations applied to
+section named by ``btf_ext_info_sec::sec_name_off``.

bpf_ext_info_sec->sec_name_off ?

Quoted 19 lines
Documentation/bpf/btf.rst +4 -0
+
+See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
+for more information on CO-RE relocations.
+
  4.2 .BTF_ids section
  --------------------
 
Documentation/bpf/llvm_reloc.rst +4 -0
index 450e6403fe3d..efe0b6ea4921 100644
@@ -240,3 +240,181 @@ The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
        Offset             Info             Type               Symbol's Value  Symbol's Name
    000000000000002c  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
    0000000000000040  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
+
+.. _btf-co-re-relocations:
+
+=================

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 256

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+CO-RE Relocations
+=================
+
+From object file point of view CO-RE mechanism is implemented as a set
+of CO-RE specific relocation records. These relocation records are not
+related to ELF relocations and are encoded in .BTF.ext section.
+See :ref:`Documentation/bpf/btf <BTF_Ext_Section>` for more
+information on .BTF.ext structure.
+
+

one empty line here?

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 268

Quoted 12 lines
Documentation/bpf/llvm_reloc.rst +12 -0
+CO-RE relocations are applied to BPF instructions to update immediate
+or offset fields of the instruction at load time with information
+relevant for target kernel.
+
+Relocation kinds
+================
+
+There are several kinds of CO-RE relocations that could be split in
+three groups:
+
+* Field-based - patch instruction with field related information, e.g.
+  change offset field of the BPF_LD instruction to reflect offset

BPF_LDX?

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 277

Quoted 9 lines
Documentation/bpf/llvm_reloc.rst +9 -0
+  of a specific structure field in the target kernel.
+
+* Type-based - patch instruction with type related information, e.g.
+  change immediate field of the BPF_MOV instruction to 0 or 1 to
+  reflect if specific type is present in the target kernel.
+
+* Enum-based - patch instruction with enum related information, e.g.
+  change immediate field of the BPF_MOV instruction to reflect value
+  of a specific enum literal in the target kernel.

BPF_MOV -> BPF_LD_IMM64 ?
below we actually have an example for this:
+ 5: r1 = 0x1 ll
+ 28: CO-RE <enumval_value> [9] enum bar::V = 1

Quoted 65 lines
Documentation/bpf/llvm_reloc.rst +65 -0
+
+The complete list of relocation kinds is represented by the following enum:
+
+.. code-block:: c
+
+ enum bpf_core_relo_kind {
+	BPF_CORE_FIELD_BYTE_OFFSET = 0,  /* field byte offset */
+	BPF_CORE_FIELD_BYTE_SIZE   = 1,  /* field size in bytes */
+	BPF_CORE_FIELD_EXISTS      = 2,  /* field existence in target kernel */
+	BPF_CORE_FIELD_SIGNED      = 3,  /* field signedness (0 - unsigned, 1 - signed) */
+	BPF_CORE_FIELD_LSHIFT_U64  = 4,  /* bitfield-specific left bitshift */
+	BPF_CORE_FIELD_RSHIFT_U64  = 5,  /* bitfield-specific right bitshift */
+	BPF_CORE_TYPE_ID_LOCAL     = 6,  /* type ID in local BPF object */
+	BPF_CORE_TYPE_ID_TARGET    = 7,  /* type ID in target kernel */
+	BPF_CORE_TYPE_EXISTS       = 8,  /* type existence in target kernel */
+	BPF_CORE_TYPE_SIZE         = 9,  /* type size in bytes */
+	BPF_CORE_ENUMVAL_EXISTS    = 10, /* enum value existence in target kernel */
+	BPF_CORE_ENUMVAL_VALUE     = 11, /* enum value integer value */
+	BPF_CORE_TYPE_MATCHES      = 12, /* type match in target kernel */
+ };
+
+CO-RE Relocation Record
+=======================
+
+Relocation record is encoded as the following structure:
+
+.. code-block:: c
+
+ struct bpf_core_relo {
+	__u32 insn_off;
+	__u32 type_id;
+	__u32 access_str_off;
+	enum bpf_core_relo_kind kind;
+ };
+
+* ``insn_off`` - instruction offset (in bytes) within a code section
+  associated with this relocation;
+
+* ``type_id`` - BTF type ID of the "root" (containing) entity of a
+  relocatable type or field;
+
+* ``access_str_off`` - offset into corresponding .BTF string section.
+  String interpretation depends on specific relocation kind:
+
+  * for field-based relocations, string encodes an accessed field using
+    a sequence of field and array indices, separated by colon (:). It's
+    conceptually very close to LLVM's `getelementptr <GEP_>`_ instruction's
+    arguments for identifying offset to a field. For example, consider the
+    following C code:
+
+    .. code-block:: c
+
+       struct sample {
+           int a;
+           int b;
+           struct { int c[10]; };
+       } __attribute__((preserve_access_index));
+       struct sample *s;
+
+    * Access to ``s[0].a`` would be encoded as ``0:0``:
+
+      * ``0``: first element of ``s`` (as if ``s`` is an array);
+      * ``0``: index of field ``a`` in ``struct sample``.
+
+    * Access to ``s->a`` would be encoded as ``0:0`` as well.

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 352

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+    * Access to ``s->b`` would be encoded as ``0:1``:
+
+      * ``0``: first element of ``s``;
+      * ``1``: index of field ``b`` in ``struct sample``.
+
+    * Access to ``s[1].c[5]`` would be encoded as ``1:2:0:5``:
+
+      * ``1``: second element of ``s``;
+      * ``2``: index of anonymous structure field in ``struct sample``;
+      * ``0``: index of field ``b`` in anonymous structure;

``b`` => ``c``

Quoted 22 lines
Documentation/bpf/llvm_reloc.rst +22 -0
+      * ``5``: access to array element #5.
+
+  * for type-based relocations, string is expected to be just "0";
+
+  * for enum value-based relocations, string contains an index of enum
+     value within its enum type;
+
+* ``kind`` - one of ``enum bpf_core_relo_kind``.
+
+.. _GEP: https://llvm.org/docs/LangRef.html#getelementptr-instruction
+
+.. _btf_co_re_relocation_examples:
+
+CO-RE Relocation Examples
+=========================
+
+For the following C code:
+
+.. code-block:: c
+
+ struct foo {
+     int a;

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 384

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+     int b;
+ } __attribute__((preserve_access_index));
+
+ enum bar { U, V };
+
+ void buz(struct foo *s, volatile unsigned long *g) {
+   s->a = 1;
+   *g = __builtin_preserve_field_info(s->b, 1);
+   *g = __builtin_preserve_type_info(*s, 1);
+   *g = __builtin_preserve_enum_value(*(enum bar *)V, 1);

Maybe __builtin_btf_type_id() can be added as well? So far, clang only supports the above 4 builtin's for core relocations.

Quoted 24 lines
Documentation/bpf/llvm_reloc.rst +24 -0
+ }
+
+With the following BTF definititions:
+
+.. code-block::
+
+ ...
+ [2] STRUCT 'foo' size=8 vlen=2
+ 	'a' type_id=3 bits_offset=0
+ 	'b' type_id=3 bits_offset=32
+ [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
+ ...
+ [9] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
+ 	'U' val=0
+ 	'V' val=1
+
+The following relocation entries would be generated:
+
+.. code-block:: c
+
+   <buz>:
+       0:	*(u32 *)(r1 + 0x0) = 0x1
+		00:  CO-RE <byte_off> [2] struct foo::a (0:0)
+       1:	r1 = 0x4

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 418

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+		08:  CO-RE <byte_sz> [2] struct foo::b (0:1)
+       2:	*(u64 *)(r2 + 0x0) = r1
+       3:	r1 = 0x8
+		18:  CO-RE <type_size> [2] struct foo
+       4:	*(u64 *)(r2 + 0x0) = r1
+       5:	r1 = 0x1 ll
+		28:  CO-RE <enumval_value> [9] enum bar::V = 1
+       7:	*(u64 *)(r2 + 0x0) = r1
+       8:	exit
+

It would be great if we can have an example for each of above core relocation kinds.

Quoted 2 lines
Documentation/bpf/llvm_reloc.rst +2 -0
+Note: modifications for llvm-objdump to show these relocation entries
+are currently work in progress.
author maintainer
#2 lore raw
Quoted 10 lines · Yonghong Song
Quoted 4 lines · Eduard Zingerman

Add a section on CO-RE relocations to llvm_relo.rst. Describe relevant .BTF.ext structure, enum bpf_core_relo_kind and struct bpf_core_relo in some detail.
Description is based on doc-string from include/uapi/linux/bpf.h.

Thanks Eduard. This is very helpful to give bpf deverlopers some insight about how different of core relocations are supported in llvm and libbpf.

Hi Yonghong,
thank you for taking a look.

Quoted 48 lines

Some comments below.

Quoted 45 lines
---
  Documentation/bpf/btf.rst        |  27 ++++-
  Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
  2 files changed, 201 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +9 -4
index f32db1f44ae9..c0530211c3c1 100644
@@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
  4.2 .BTF.ext section
  --------------------
 
-The .BTF.ext section encodes func_info and line_info which needs loader
-manipulation before loading into the kernel.
+The .BTF.ext section encodes func_info, line_info and CO-RE relocations
+which needs loader manipulation before loading into the kernel.
 
  The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
  and ``tools/lib/bpf/btf.c``.
@@ -745,11 +745,16 @@ The current header of .BTF.ext section::
          __u32   func_info_len;
          __u32   line_info_off;
          __u32   line_info_len;
+
+        /* optional part of .BTF.ext header */
+        __u32   core_relo_off;
+        __u32   core_relo_len;
      };
 
  It is very similar to .BTF section. Instead of type/string section, it
-contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
-about func_info and line_info record format.
+contains func_info, line_info and core_relo sub-sections.
+See :ref:`BPF_Prog_Load` for details about func_info and line_info
+record format.
 
  The func_info is organized as below.::
 
@@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
  bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
  beginning of section (``btf_ext_info_sec->sec_name_off``).
 
Quoted 12 lines

Documentation/bpf/btf.rst kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct · line 804

Quoted 10 lines
Documentation/bpf/btf.rst +10 -0
+The core_relo is organized as below.::
+
+     core_relo_rec_size
+     btf_ext_info_sec for section #1 /* core_relo for section #1 */
+     btf_ext_info_sec for section #2 /* core_relo for section #2 */
+
+``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
+structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
+within a single ``btf_ext_info_sec`` describe relocations applied to
+section named by ``btf_ext_info_sec::sec_name_off``.

bpf_ext_info_sec->sec_name_off ?

Will change.

Quoted 24 lines
Quoted 24 lines
Documentation/bpf/btf.rst +4 -0
+
+See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
+for more information on CO-RE relocations.
+
  4.2 .BTF_ids section
  --------------------
 
Documentation/bpf/llvm_reloc.rst +8 -0
index 450e6403fe3d..efe0b6ea4921 100644
@@ -240,3 +240,181 @@ The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
        Offset             Info             Type               Symbol's Value  Symbol's Name
    000000000000002c  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
    0000000000000040  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
+
+.. _btf-co-re-relocations:
+
+=================
+CO-RE Relocations
+=================
+
+From object file point of view CO-RE mechanism is implemented as a set
Quoted 8 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 256

Quoted 6 lines
Documentation/bpf/llvm_reloc.rst +6 -0
+of CO-RE specific relocation records. These relocation records are not
+related to ELF relocations and are encoded in .BTF.ext section.
+See :ref:`Documentation/bpf/btf <BTF_Ext_Section>` for more
+information on .BTF.ext structure.
+
+

one empty line here?

Will change.

Quoted 15 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 268

Quoted 13 lines
Documentation/bpf/llvm_reloc.rst +12 -0
+CO-RE relocations are applied to BPF instructions to update immediate
+or offset fields of the instruction at load time with information
+relevant for target kernel.
+
+Relocation kinds
+================
+
+There are several kinds of CO-RE relocations that could be split in
+three groups:
+
+* Field-based - patch instruction with field related information, e.g.
+  change offset field of the BPF_LD instruction to reflect offset

BPF_LDX?

Correct, thank you.

Quoted 15 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 277

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +9 -0
+  of a specific structure field in the target kernel.
+
+* Type-based - patch instruction with type related information, e.g.
+  change immediate field of the BPF_MOV instruction to 0 or 1 to
+  reflect if specific type is present in the target kernel.
+
+* Enum-based - patch instruction with enum related information, e.g.
+  change immediate field of the BPF_MOV instruction to reflect value
+  of a specific enum literal in the target kernel.

BPF_MOV -> BPF_LD_IMM64 ?
below we actually have an example for this:
+ 5: r1 = 0x1 ll
+ 28: CO-RE <enumval_value> [9] enum bar::V = 1

Correct, thank you.

Quoted 71 lines
Quoted 71 lines
Documentation/bpf/llvm_reloc.rst +70 -0
+
+The complete list of relocation kinds is represented by the following enum:
+
+.. code-block:: c
+
+ enum bpf_core_relo_kind {
+	BPF_CORE_FIELD_BYTE_OFFSET = 0,  /* field byte offset */
+	BPF_CORE_FIELD_BYTE_SIZE   = 1,  /* field size in bytes */
+	BPF_CORE_FIELD_EXISTS      = 2,  /* field existence in target kernel */
+	BPF_CORE_FIELD_SIGNED      = 3,  /* field signedness (0 - unsigned, 1 - signed) */
+	BPF_CORE_FIELD_LSHIFT_U64  = 4,  /* bitfield-specific left bitshift */
+	BPF_CORE_FIELD_RSHIFT_U64  = 5,  /* bitfield-specific right bitshift */
+	BPF_CORE_TYPE_ID_LOCAL     = 6,  /* type ID in local BPF object */
+	BPF_CORE_TYPE_ID_TARGET    = 7,  /* type ID in target kernel */
+	BPF_CORE_TYPE_EXISTS       = 8,  /* type existence in target kernel */
+	BPF_CORE_TYPE_SIZE         = 9,  /* type size in bytes */
+	BPF_CORE_ENUMVAL_EXISTS    = 10, /* enum value existence in target kernel */
+	BPF_CORE_ENUMVAL_VALUE     = 11, /* enum value integer value */
+	BPF_CORE_TYPE_MATCHES      = 12, /* type match in target kernel */
+ };
+
+CO-RE Relocation Record
+=======================
+
+Relocation record is encoded as the following structure:
+
+.. code-block:: c
+
+ struct bpf_core_relo {
+	__u32 insn_off;
+	__u32 type_id;
+	__u32 access_str_off;
+	enum bpf_core_relo_kind kind;
+ };
+
+* ``insn_off`` - instruction offset (in bytes) within a code section
+  associated with this relocation;
+
+* ``type_id`` - BTF type ID of the "root" (containing) entity of a
+  relocatable type or field;
+
+* ``access_str_off`` - offset into corresponding .BTF string section.
+  String interpretation depends on specific relocation kind:
+
+  * for field-based relocations, string encodes an accessed field using
+    a sequence of field and array indices, separated by colon (:). It's
+    conceptually very close to LLVM's `getelementptr <GEP_>`_ instruction's
+    arguments for identifying offset to a field. For example, consider the
+    following C code:
+
+    .. code-block:: c
+
+       struct sample {
+           int a;
+           int b;
+           struct { int c[10]; };
+       } __attribute__((preserve_access_index));
+       struct sample *s;
+
+    * Access to ``s[0].a`` would be encoded as ``0:0``:
+
+      * ``0``: first element of ``s`` (as if ``s`` is an array);
+      * ``0``: index of field ``a`` in ``struct sample``.
+
+    * Access to ``s->a`` would be encoded as ``0:0`` as well.
+    * Access to ``s->b`` would be encoded as ``0:1``:
+
+      * ``0``: first element of ``s``;
+      * ``1``: index of field ``b`` in ``struct sample``.
+
Quoted 8 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 352

Quoted 5 lines
Documentation/bpf/llvm_reloc.rst +5 -0
+    * Access to ``s[1].c[5]`` would be encoded as ``1:2:0:5``:
+
+      * ``1``: second element of ``s``;
+      * ``2``: index of anonymous structure field in ``struct sample``;
+      * ``0``: index of field ``b`` in anonymous structure;

``b`` => ``c``

Right, sorry, changed the example a few times ...

Quoted 29 lines
Quoted 29 lines
Documentation/bpf/llvm_reloc.rst +28 -0
+      * ``5``: access to array element #5.
+
+  * for type-based relocations, string is expected to be just "0";
+
+  * for enum value-based relocations, string contains an index of enum
+     value within its enum type;
+
+* ``kind`` - one of ``enum bpf_core_relo_kind``.
+
+.. _GEP: https://llvm.org/docs/LangRef.html#getelementptr-instruction
+
+.. _btf_co_re_relocation_examples:
+
+CO-RE Relocation Examples
+=========================
+
+For the following C code:
+
+.. code-block:: c
+
+ struct foo {
+     int a;
+     int b;
+ } __attribute__((preserve_access_index));
+
+ enum bar { U, V };
+
+ void buz(struct foo *s, volatile unsigned long *g) {
Quoted 8 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 384

Quoted 4 lines
Documentation/bpf/llvm_reloc.rst +4 -0
+   s->a = 1;
+   *g = __builtin_preserve_field_info(s->b, 1);
+   *g = __builtin_preserve_type_info(*s, 1);
+   *g = __builtin_preserve_enum_value(*(enum bar *)V, 1);

Maybe __builtin_btf_type_id() can be added as well? So far, clang only supports the above 4 builtin's for core relocations.

Will add __builtin_btf_type_id() as well.

Quoted 30 lines
Quoted 30 lines
Documentation/bpf/llvm_reloc.rst +29 -0
+ }
+
+With the following BTF definititions:
+
+.. code-block::
+
+ ...
+ [2] STRUCT 'foo' size=8 vlen=2
+ 	'a' type_id=3 bits_offset=0
+ 	'b' type_id=3 bits_offset=32
+ [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
+ ...
+ [9] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
+ 	'U' val=0
+ 	'V' val=1
+
+The following relocation entries would be generated:
+
+.. code-block:: c
+
+   <buz>:
+       0:	*(u32 *)(r1 + 0x0) = 0x1
+		00:  CO-RE <byte_off> [2] struct foo::a (0:0)
+       1:	r1 = 0x4
+		08:  CO-RE <byte_sz> [2] struct foo::b (0:1)
+       2:	*(u64 *)(r2 + 0x0) = r1
+       3:	r1 = 0x8
+		18:  CO-RE <type_size> [2] struct foo
+       4:	*(u64 *)(r2 + 0x0) = r1
Quoted 8 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 418

Quoted 5 lines
Documentation/bpf/llvm_reloc.rst +5 -0
+       5:	r1 = 0x1 ll
+		28:  CO-RE <enumval_value> [9] enum bar::V = 1
+       7:	*(u64 *)(r2 + 0x0) = r1
+       8:	exit
+

It would be great if we can have an example for each of above core relocation kinds.

You mean all 13 kinds, right?

Quoted 3 lines
Quoted 3 lines
Documentation/bpf/llvm_reloc.rst +2 -0
+Note: modifications for llvm-objdump to show these relocation entries
+are currently work in progress.
reviewer
#3 lore raw
Quoted 32 lines · Eduard Zingerman
Quoted 10 lines · Yonghong Song
Quoted 4 lines · Eduard Zingerman

Add a section on CO-RE relocations to llvm_relo.rst. Describe relevant .BTF.ext structure, enum bpf_core_relo_kind and struct bpf_core_relo in some detail.
Description is based on doc-string from include/uapi/linux/bpf.h.

Thanks Eduard. This is very helpful to give bpf deverlopers some insight about how different of core relocations are supported in llvm and libbpf.

Hi Yonghong,
thank you for taking a look.

Quoted 17 lines

Some comments below.

Quoted 14 lines
---
   Documentation/bpf/btf.rst        |  27 ++++-
   Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
   2 files changed, 201 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +0 -0
index f32db1f44ae9..c0530211c3c1 100644
@@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
   4.2 .BTF.ext section
   --------------------

[...]

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Documentation/bpf/btf.rst +21 -0
+
+The complete list of relocation kinds is represented by the following enum:
+
+.. code-block:: c
+
+ enum bpf_core_relo_kind {
+	BPF_CORE_FIELD_BYTE_OFFSET = 0,  /* field byte offset */
+	BPF_CORE_FIELD_BYTE_SIZE   = 1,  /* field size in bytes */
+	BPF_CORE_FIELD_EXISTS      = 2,  /* field existence in target kernel */
+	BPF_CORE_FIELD_SIGNED      = 3,  /* field signedness (0 - unsigned, 1 - signed) */
+	BPF_CORE_FIELD_LSHIFT_U64  = 4,  /* bitfield-specific left bitshift */
+	BPF_CORE_FIELD_RSHIFT_U64  = 5,  /* bitfield-specific right bitshift */
+	BPF_CORE_TYPE_ID_LOCAL     = 6,  /* type ID in local BPF object */
+	BPF_CORE_TYPE_ID_TARGET    = 7,  /* type ID in target kernel */
+	BPF_CORE_TYPE_EXISTS       = 8,  /* type existence in target kernel */
+	BPF_CORE_TYPE_SIZE         = 9,  /* type size in bytes */
+	BPF_CORE_ENUMVAL_EXISTS    = 10, /* enum value existence in target kernel */
+	BPF_CORE_ENUMVAL_VALUE     = 11, /* enum value integer value */
+	BPF_CORE_TYPE_MATCHES      = 12, /* type match in target kernel */
+ };
+

[...]

Quoted 66 lines
Quoted 16 lines
Quoted 16 lines
Documentation/bpf/btf.rst +16 -0
+
+CO-RE Relocation Examples
+=========================
+
+For the following C code:
+
+.. code-block:: c
+
+ struct foo {
+     int a;
+     int b;
+ } __attribute__((preserve_access_index));
+
+ enum bar { U, V };
+
+ void buz(struct foo *s, volatile unsigned long *g) {
Quoted 8 lines

Documentation/bpf/btf.rst

Quoted 4 lines
Documentation/bpf/btf.rst +4 -0
+   s->a = 1;
+   *g = __builtin_preserve_field_info(s->b, 1);
+   *g = __builtin_preserve_type_info(*s, 1);
+   *g = __builtin_preserve_enum_value(*(enum bar *)V, 1);

Maybe __builtin_btf_type_id() can be added as well? So far, clang only supports the above 4 builtin's for core relocations.

Will add __builtin_btf_type_id() as well.

Quoted 30 lines
Quoted 30 lines

+ }

Documentation/bpf/btf.rst +28 -0
+
+With the following BTF definititions:
+
+.. code-block::
+
+ ...
+ [2] STRUCT 'foo' size=8 vlen=2
+ 	'a' type_id=3 bits_offset=0
+ 	'b' type_id=3 bits_offset=32
+ [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
+ ...
+ [9] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
+ 	'U' val=0
+ 	'V' val=1
+
+The following relocation entries would be generated:
+
+.. code-block:: c
+
+   <buz>:
+       0:	*(u32 *)(r1 + 0x0) = 0x1
+		00:  CO-RE <byte_off> [2] struct foo::a (0:0)
+       1:	r1 = 0x4
+		08:  CO-RE <byte_sz> [2] struct foo::b (0:1)
+       2:	*(u64 *)(r2 + 0x0) = r1
+       3:	r1 = 0x8
+		18:  CO-RE <type_size> [2] struct foo
+       4:	*(u64 *)(r2 + 0x0) = r1
Quoted 8 lines

Documentation/bpf/btf.rst

Quoted 5 lines
Documentation/bpf/btf.rst +5 -0
+       5:	r1 = 0x1 ll
+		28:  CO-RE <enumval_value> [9] enum bar::V = 1
+       7:	*(u64 *)(r2 + 0x0) = r1
+       8:	exit
+

It would be great if we can have an example for each of above core relocation kinds.

Quoted 1 line

You mean all 13 kinds, right?

Yes, it would be great if we have at least one example for each kind to illustrate what this relo kind intends to do.

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Documentation/bpf/btf.rst +2 -0
+Note: modifications for llvm-objdump to show these relocation entries
+are currently work in progress.
#4 lore raw
In reply to Eduard Zingerman
Quoted 8 lines · Eduard Zingerman

Add a section on CO-RE relocations to llvm_relo.rst. Describe relevant .BTF.ext structure, enum bpf_core_relo_kind and struct bpf_core_relo in some detail.
Description is based on doc-string from include/uapi/linux/bpf.h.

---

Looks great overall, thanks a lot for adding this!

Quoted 39 lines
 Documentation/bpf/btf.rst        |  27 ++++-
 Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
 2 files changed, 201 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +9 -4
index f32db1f44ae9..c0530211c3c1 100644
@@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
 4.2 .BTF.ext section
 --------------------
 
-The .BTF.ext section encodes func_info and line_info which needs loader
-manipulation before loading into the kernel.
+The .BTF.ext section encodes func_info, line_info and CO-RE relocations
+which needs loader manipulation before loading into the kernel.
 
 The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
 and ``tools/lib/bpf/btf.c``.
@@ -745,11 +745,16 @@ The current header of .BTF.ext section::
         __u32   func_info_len;
         __u32   line_info_off;
         __u32   line_info_len;
+
+        /* optional part of .BTF.ext header */
+        __u32   core_relo_off;
+        __u32   core_relo_len;
     };
 
 It is very similar to .BTF section. Instead of type/string section, it
-contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
-about func_info and line_info record format.
+contains func_info, line_info and core_relo sub-sections.
+See :ref:`BPF_Prog_Load` for details about func_info and line_info
+record format.
 
 The func_info is organized as below.::
 

Documentation/bpf/btf.rst kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct · line 797

Quoted 6 lines
Documentation/bpf/btf.rst +3 -0
@@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
 bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
 beginning of section (``btf_ext_info_sec->sec_name_off``).
 
+The core_relo is organized as below.::
+
+     core_relo_rec_size

nit: should we specify that this is __u32 value? Same for func_info and line_info. I'm not sure we ever explicitly mention this this record size is 4 byte long.

Quoted 48 lines
Documentation/bpf/btf.rst +11 -0
+     btf_ext_info_sec for section #1 /* core_relo for section #1 */
+     btf_ext_info_sec for section #2 /* core_relo for section #2 */
+
+``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
+structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
+within a single ``btf_ext_info_sec`` describe relocations applied to
+section named by ``btf_ext_info_sec::sec_name_off``.
+
+See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
+for more information on CO-RE relocations.
+
 4.2 .BTF_ids section
 --------------------
 
Documentation/bpf/llvm_reloc.rst +26 -0
index 450e6403fe3d..efe0b6ea4921 100644
@@ -240,3 +240,181 @@ The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
       Offset             Info             Type               Symbol's Value  Symbol's Name
   000000000000002c  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
   0000000000000040  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
+
+.. _btf-co-re-relocations:
+
+=================
+CO-RE Relocations
+=================
+
+From object file point of view CO-RE mechanism is implemented as a set
+of CO-RE specific relocation records. These relocation records are not
+related to ELF relocations and are encoded in .BTF.ext section.
+See :ref:`Documentation/bpf/btf <BTF_Ext_Section>` for more
+information on .BTF.ext structure.
+
+
+CO-RE relocations are applied to BPF instructions to update immediate
+or offset fields of the instruction at load time with information
+relevant for target kernel.
+
+Relocation kinds
+================
+
+There are several kinds of CO-RE relocations that could be split in
+three groups:
+
+* Field-based - patch instruction with field related information, e.g.
+  change offset field of the BPF_LD instruction to reflect offset

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 278

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+  of a specific structure field in the target kernel.
+
+* Type-based - patch instruction with type related information, e.g.
+  change immediate field of the BPF_MOV instruction to 0 or 1 to
+  reflect if specific type is present in the target kernel.
+
+* Enum-based - patch instruction with enum related information, e.g.
+  change immediate field of the BPF_MOV instruction to reflect value
+  of a specific enum literal in the target kernel.
+

Instead of referencing BPF_MOV specifically, would it be useful to incorporate all the different instructions that can be relocated? bpf_core_patch_insn comment has a nice summary, maybe we can somehow reuse it in this doc as well?

* Currently supported classes of BPF instruction are:
* 1. rX = <imm> (assignment with immediate operand);
* 2. rX += <imm> (arithmetic operations with immediate operand);
* 3. rX = <imm64> (load with 64-bit immediate value);
* 4. rX = *(T *)(rY + <off>), where T is one of {u8, u16, u32, u64};
* 5. *(T *)(rX + <off>) = rY, where T is one of {u8, u16, u32, u64};
* 6. *(T *)(rX + <off>) = <imm>, where T is one of {u8, u16, u32, u64}.

Quoted 23 lines
Documentation/bpf/llvm_reloc.rst +23 -0
+The complete list of relocation kinds is represented by the following enum:
+
+.. code-block:: c
+
+ enum bpf_core_relo_kind {
+       BPF_CORE_FIELD_BYTE_OFFSET = 0,  /* field byte offset */
+       BPF_CORE_FIELD_BYTE_SIZE   = 1,  /* field size in bytes */
+       BPF_CORE_FIELD_EXISTS      = 2,  /* field existence in target kernel */
+       BPF_CORE_FIELD_SIGNED      = 3,  /* field signedness (0 - unsigned, 1 - signed) */
+       BPF_CORE_FIELD_LSHIFT_U64  = 4,  /* bitfield-specific left bitshift */
+       BPF_CORE_FIELD_RSHIFT_U64  = 5,  /* bitfield-specific right bitshift */
+       BPF_CORE_TYPE_ID_LOCAL     = 6,  /* type ID in local BPF object */
+       BPF_CORE_TYPE_ID_TARGET    = 7,  /* type ID in target kernel */
+       BPF_CORE_TYPE_EXISTS       = 8,  /* type existence in target kernel */
+       BPF_CORE_TYPE_SIZE         = 9,  /* type size in bytes */
+       BPF_CORE_ENUMVAL_EXISTS    = 10, /* enum value existence in target kernel */
+       BPF_CORE_ENUMVAL_VALUE     = 11, /* enum value integer value */
+       BPF_CORE_TYPE_MATCHES      = 12, /* type match in target kernel */
+ };
+
+CO-RE Relocation Record
+=======================
+

[...]

Quoted 12 lines
Documentation/bpf/llvm_reloc.rst +12 -0
+CO-RE Relocation Examples
+=========================
+
+For the following C code:
+
+.. code-block:: c
+
+ struct foo {
+     int a;
+     int b;
+ } __attribute__((preserve_access_index));
+

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 387

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+ enum bar { U, V };
+
+ void buz(struct foo *s, volatile unsigned long *g) {
+   s->a = 1;
+   *g = __builtin_preserve_field_info(s->b, 1);
+   *g = __builtin_preserve_type_info(*s, 1);
+   *g = __builtin_preserve_enum_value(*(enum bar *)V, 1);
+ }
+
+With the following BTF definititions:

Gmail points to typo in "definitions"

Quoted 23 lines
Documentation/bpf/llvm_reloc.rst +23 -0
+
+.. code-block::
+
+ ...
+ [2] STRUCT 'foo' size=8 vlen=2
+       'a' type_id=3 bits_offset=0
+       'b' type_id=3 bits_offset=32
+ [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
+ ...
+ [9] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
+       'U' val=0
+       'V' val=1
+
+The following relocation entries would be generated:
+
+.. code-block:: c
+
+   <buz>:
+       0:      *(u32 *)(r1 + 0x0) = 0x1
+               00:  CO-RE <byte_off> [2] struct foo::a (0:0)
+       1:      r1 = 0x4
+               08:  CO-RE <byte_sz> [2] struct foo::b (0:1)
+       2:      *(u64 *)(r2 + 0x0) = r1

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 420

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+       3:      r1 = 0x8
+               18:  CO-RE <type_size> [2] struct foo
+       4:      *(u64 *)(r2 + 0x0) = r1
+       5:      r1 = 0x1 ll
+               28:  CO-RE <enumval_value> [9] enum bar::V = 1
+       7:      *(u64 *)(r2 + 0x0) = r1
+       8:      exit
+
+Note: modifications for llvm-objdump to show these relocation entries
+are currently work in progress.

Do we need this note here? Doesn't seem like you have any other reference to llvm-objdump?

Quoted 3 lines

--
2.41.0

author maintainer
#5 lore raw
Quoted 54 lines · Andrii Nakryiko
Quoted 8 lines · Eduard Zingerman

Add a section on CO-RE relocations to llvm_relo.rst. Describe relevant .BTF.ext structure, enum bpf_core_relo_kind and struct bpf_core_relo in some detail.
Description is based on doc-string from include/uapi/linux/bpf.h.

---

Looks great overall, thanks a lot for adding this!

Quoted 42 lines
 Documentation/bpf/btf.rst        |  27 ++++-
 Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
 2 files changed, 201 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +9 -4
index f32db1f44ae9..c0530211c3c1 100644
@@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
 4.2 .BTF.ext section
 --------------------
 
-The .BTF.ext section encodes func_info and line_info which needs loader
-manipulation before loading into the kernel.
+The .BTF.ext section encodes func_info, line_info and CO-RE relocations
+which needs loader manipulation before loading into the kernel.
 
 The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
 and ``tools/lib/bpf/btf.c``.
@@ -745,11 +745,16 @@ The current header of .BTF.ext section::
         __u32   func_info_len;
         __u32   line_info_off;
         __u32   line_info_len;
+
+        /* optional part of .BTF.ext header */
+        __u32   core_relo_off;
+        __u32   core_relo_len;
     };
 
 It is very similar to .BTF section. Instead of type/string section, it
-contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
-about func_info and line_info record format.
+contains func_info, line_info and core_relo sub-sections.
+See :ref:`BPF_Prog_Load` for details about func_info and line_info
+record format.
 
 The func_info is organized as below.::
 
@@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
 bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
 beginning of section (``btf_ext_info_sec->sec_name_off``).
 
Quoted 7 lines

Documentation/bpf/btf.rst kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct · line 797

Quoted 3 lines
Documentation/bpf/btf.rst +3 -0
+The core_relo is organized as below.::
+
+     core_relo_rec_size

nit: should we specify that this is __u32 value? Same for func_info and line_info. I'm not sure we ever explicitly mention this this record size is 4 byte long.

Yeap, we don't mention it anywhere, I will add __u32 to the description.

Quoted 65 lines
Quoted 49 lines
Documentation/bpf/btf.rst +11 -0
+     btf_ext_info_sec for section #1 /* core_relo for section #1 */
+     btf_ext_info_sec for section #2 /* core_relo for section #2 */
+
+``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
+structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
+within a single ``btf_ext_info_sec`` describe relocations applied to
+section named by ``btf_ext_info_sec::sec_name_off``.
+
+See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
+for more information on CO-RE relocations.
+
 4.2 .BTF_ids section
 --------------------
 
Documentation/bpf/llvm_reloc.rst +26 -0
index 450e6403fe3d..efe0b6ea4921 100644
@@ -240,3 +240,181 @@ The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
       Offset             Info             Type               Symbol's Value  Symbol's Name
   000000000000002c  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
   0000000000000040  0000000200000004 R_BPF_64_NODYLD32      0000000000000000 .text
+
+.. _btf-co-re-relocations:
+
+=================
+CO-RE Relocations
+=================
+
+From object file point of view CO-RE mechanism is implemented as a set
+of CO-RE specific relocation records. These relocation records are not
+related to ELF relocations and are encoded in .BTF.ext section.
+See :ref:`Documentation/bpf/btf <BTF_Ext_Section>` for more
+information on .BTF.ext structure.
+
+
+CO-RE relocations are applied to BPF instructions to update immediate
+or offset fields of the instruction at load time with information
+relevant for target kernel.
+
+Relocation kinds
+================
+
+There are several kinds of CO-RE relocations that could be split in
+three groups:
+
+* Field-based - patch instruction with field related information, e.g.
+  change offset field of the BPF_LD instruction to reflect offset

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 278

Quoted 10 lines
Documentation/bpf/llvm_reloc.rst +10 -0
+  of a specific structure field in the target kernel.
+
+* Type-based - patch instruction with type related information, e.g.
+  change immediate field of the BPF_MOV instruction to 0 or 1 to
+  reflect if specific type is present in the target kernel.
+
+* Enum-based - patch instruction with enum related information, e.g.
+  change immediate field of the BPF_MOV instruction to reflect value
+  of a specific enum literal in the target kernel.
+

Instead of referencing BPF_MOV specifically, would it be useful to incorporate all the different instructions that can be relocated? bpf_core_patch_insn comment has a nice summary, maybe we can somehow reuse it in this doc as well?

Quoted 7 lines

* Currently supported classes of BPF instruction are:
* 1. rX = <imm> (assignment with immediate operand);
* 2. rX += <imm> (arithmetic operations with immediate operand);
* 3. rX = <imm64> (load with 64-bit immediate value);
* 4. rX = *(T *)(rY + <off>), where T is one of {u8, u16, u32, u64};
* 5. *(T *)(rX + <off>) = rY, where T is one of {u8, u16, u32, u64};
* 6. *(T *)(rX + <off>) = <imm>, where T is one of {u8, u16, u32, u64}.

Good point. I will keep the BPF_MOV as an example for relocation kind groups description and add this comment describing all relocatable instructions.

Quoted 42 lines
Quoted 23 lines
Documentation/bpf/llvm_reloc.rst +23 -0
+The complete list of relocation kinds is represented by the following enum:
+
+.. code-block:: c
+
+ enum bpf_core_relo_kind {
+       BPF_CORE_FIELD_BYTE_OFFSET = 0,  /* field byte offset */
+       BPF_CORE_FIELD_BYTE_SIZE   = 1,  /* field size in bytes */
+       BPF_CORE_FIELD_EXISTS      = 2,  /* field existence in target kernel */
+       BPF_CORE_FIELD_SIGNED      = 3,  /* field signedness (0 - unsigned, 1 - signed) */
+       BPF_CORE_FIELD_LSHIFT_U64  = 4,  /* bitfield-specific left bitshift */
+       BPF_CORE_FIELD_RSHIFT_U64  = 5,  /* bitfield-specific right bitshift */
+       BPF_CORE_TYPE_ID_LOCAL     = 6,  /* type ID in local BPF object */
+       BPF_CORE_TYPE_ID_TARGET    = 7,  /* type ID in target kernel */
+       BPF_CORE_TYPE_EXISTS       = 8,  /* type existence in target kernel */
+       BPF_CORE_TYPE_SIZE         = 9,  /* type size in bytes */
+       BPF_CORE_ENUMVAL_EXISTS    = 10, /* enum value existence in target kernel */
+       BPF_CORE_ENUMVAL_VALUE     = 11, /* enum value integer value */
+       BPF_CORE_TYPE_MATCHES      = 12, /* type match in target kernel */
+ };
+
+CO-RE Relocation Record
+=======================
+

[...]

Quoted 16 lines
Documentation/bpf/llvm_reloc.rst +16 -0
+CO-RE Relocation Examples
+=========================
+
+For the following C code:
+
+.. code-block:: c
+
+ struct foo {
+     int a;
+     int b;
+ } __attribute__((preserve_access_index));
+
+ enum bar { U, V };
+
+ void buz(struct foo *s, volatile unsigned long *g) {
+   s->a = 1;
Quoted 8 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 387

Quoted 6 lines
Documentation/bpf/llvm_reloc.rst +6 -0
+   *g = __builtin_preserve_field_info(s->b, 1);
+   *g = __builtin_preserve_type_info(*s, 1);
+   *g = __builtin_preserve_enum_value(*(enum bar *)V, 1);
+ }
+
+With the following BTF definititions:

Gmail points to typo in "definitions"

Ouch.

Quoted 28 lines
Quoted 28 lines
Documentation/bpf/llvm_reloc.rst +28 -0
+
+.. code-block::
+
+ ...
+ [2] STRUCT 'foo' size=8 vlen=2
+       'a' type_id=3 bits_offset=0
+       'b' type_id=3 bits_offset=32
+ [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
+ ...
+ [9] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
+       'U' val=0
+       'V' val=1
+
+The following relocation entries would be generated:
+
+.. code-block:: c
+
+   <buz>:
+       0:      *(u32 *)(r1 + 0x0) = 0x1
+               00:  CO-RE <byte_off> [2] struct foo::a (0:0)
+       1:      r1 = 0x4
+               08:  CO-RE <byte_sz> [2] struct foo::b (0:1)
+       2:      *(u64 *)(r2 + 0x0) = r1
+       3:      r1 = 0x8
+               18:  CO-RE <type_size> [2] struct foo
+       4:      *(u64 *)(r2 + 0x0) = r1
+       5:      r1 = 0x1 ll
+               28:  CO-RE <enumval_value> [9] enum bar::V = 1
Quoted 8 lines

Documentation/bpf/llvm_reloc.rst The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations:: · line 420

Quoted 5 lines
Documentation/bpf/llvm_reloc.rst +5 -0
+       7:      *(u64 *)(r2 + 0x0) = r1
+       8:      exit
+
+Note: modifications for llvm-objdump to show these relocation entries
+are currently work in progress.

Do we need this note here? Doesn't seem like you have any other reference to llvm-objdump?

Idk, I hope that llvm-objdump update would be merged eventually and I'll replace the note with correct llvm-objdump incantation.

author maintainer
#6 lore raw

On Sat, 2023-08-26 at 00:10 +0300, Eduard Zingerman wrote:
[...]

Quoted 14 lines
Quoted 6 lines

Instead of referencing BPF_MOV specifically, would it be useful to incorporate all the different instructions that can be relocated? bpf_core_patch_insn comment has a nice summary, maybe we can somehow reuse it in this doc as well?

Quoted 7 lines

* Currently supported classes of BPF instruction are:
* 1. rX = <imm> (assignment with immediate operand);
* 2. rX += <imm> (arithmetic operations with immediate operand);
* 3. rX = <imm64> (load with 64-bit immediate value);
* 4. rX = *(T *)(rY + <off>), where T is one of {u8, u16, u32, u64};
* 5. *(T *)(rX + <off>) = rY, where T is one of {u8, u16, u32, u64};
* 6. *(T *)(rX + <off>) = <imm>, where T is one of {u8, u16, u32, u64}.

Quoted 4 lines

Good point. I will keep the BPF_MOV as an example for relocation kind groups description and add this comment describing all relocatable instructions.

Actually, this comment does not mention atomic instructions or loads with sign extension. bpf_core_patch_insn() operates basing on instruction class. I'll describe it as follows:

Field to patch is selected basing on the instruction class:

* For BPF_ALU, BPF_ALU64, BPF_LD immediate field is patched;
* For BPF_LDX, BPF_STX, BPF_ST offset field is patched;

WDYT?

[...]

#7 lore raw
Quoted 26 lines · Eduard Zingerman

On Sat, 2023-08-26 at 00:10 +0300, Eduard Zingerman wrote:
[...]

Quoted 14 lines
Quoted 6 lines

Instead of referencing BPF_MOV specifically, would it be useful to incorporate all the different instructions that can be relocated? bpf_core_patch_insn comment has a nice summary, maybe we can somehow reuse it in this doc as well?

Quoted 7 lines

* Currently supported classes of BPF instruction are:
* 1. rX = <imm> (assignment with immediate operand);
* 2. rX += <imm> (arithmetic operations with immediate operand);
* 3. rX = <imm64> (load with 64-bit immediate value);
* 4. rX = *(T *)(rY + <off>), where T is one of {u8, u16, u32, u64};
* 5. *(T *)(rX + <off>) = rY, where T is one of {u8, u16, u32, u64};
* 6. *(T *)(rX + <off>) = <imm>, where T is one of {u8, u16, u32, u64}.

Quoted 4 lines

Good point. I will keep the BPF_MOV as an example for relocation kind groups description and add this comment describing all relocatable instructions.

Actually, this comment does not mention atomic instructions or loads with sign extension. bpf_core_patch_insn() operates basing on instruction class. I'll describe it as follows:

Quoted 6 lines

Field to patch is selected basing on the instruction class:

* For BPF_ALU, BPF_ALU64, BPF_LD immediate field is patched;
* For BPF_LDX, BPF_STX, BPF_ST offset field is patched;

WDYT?

SGTM. As a human I still like the rX += <imm> notation, but for documentation BPF_ALU would be more precise and appropriate.

Quoted 2 lines

[...]