Skip to contentSkip to conversation

kernel: Commit without running hooks

PATCH In review William A. Kennington III 4 participants 9 messages Oct 25, 2023 – Oct 28, 2023 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kernel: Commit without running hooks

The hooks are pulled from the impure environment and are often broken in our environments. There is no reason to add extra metadata or verify the commit message as its arbitrary to turn the tarball into a git repo.

---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -1
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
 		git init
 		check_git_config
 		git add .
-		git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+		git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
 		git clean -d -f
 	fi
 
Signature · 1 line
2.42.0.820.g83a721a137-goog
#1 lore raw
Quoted 4 lines · William A. Kennington III via lists.openembedded.org

The hooks are pulled from the impure environment and are often broken in our environments. There is no reason to add extra metadata or verify the commit message as its arbitrary to turn the tarball into a git repo.

But what about the other uses of git ? If the hooks are broken during the creation step, other uses of git should also be broken.

We've had the ability to check the git config for quite some time, we could take it further an inhibit the use of the host config if we are worried about breakage like this.

Bruce

Quoted 23 lines
---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -2
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
                git init
                check_git_config
                git add .
-               git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+               git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
                git clean -d -f
        fi
 
--

2.42.0.820.g83a721a137-goog

Quoted 8 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189694): https://lists.openembedded.org/g/openembedded-core/message/189694
Mute This Topic: https://lists.openembedded.org/mt/102189231/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#2 lore raw
Quoted 10 lines · Bruce Ashfield
Quoted 4 lines · William A. Kennington III via lists.openembedded.org

The hooks are pulled from the impure environment and are often broken in our environments. There is no reason to add extra metadata or verify the commit message as its arbitrary to turn the tarball into a git repo.

But what about the other uses of git ? If the hooks are broken during the creation step, other uses of git should also be broken.

The only time hooks are run is for git commit. AFAIK our build process has no other invocations of git commit. All other git operations work fine.

Quoted 4 lines

We've had the ability to check the git config for quite some time, we could take it further an inhibit the use of the host config if we are worried about breakage like this.

Maybe this would be the best idea overall for our git support inside the build system, avoiding the build user global git config. That probably solves the issue we generally have. I'm okay either way.

Quoted 40 lines

Bruce

Quoted 23 lines
---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -2
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
                git init
                check_git_config
                git add .
-               git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+               git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
                git clean -d -f
        fi
 
--

2.42.0.820.g83a721a137-goog

Quoted 8 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189694): https://lists.openembedded.org/g/openembedded-core/message/189694
Mute This Topic: https://lists.openembedded.org/mt/102189231/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#3 lore raw
Quoted 10 lines · William Kennington
Quoted 8 lines · Bruce Ashfield
Quoted 4 lines · William A. Kennington III via lists.openembedded.org

The hooks are pulled from the impure environment and are often broken in our environments. There is no reason to add extra metadata or verify the commit message as its arbitrary to turn the tarball into a git repo.

Quoted 6 lines
Quoted 2 lines

But what about the other uses of git ? If the hooks are broken during the creation step, other uses of git should also be broken.

The only time hooks are run is for git commit. AFAIK our build process has no other invocations of git commit. All other git operations work fine.

Right. There are a few others that you just aren't hitting. There's the git option for patching (for any recipe), and also some use in the kern-tools that you wouldn't see unless you are also patching or doing merges as part of the kernel patch phase.

It would be nice to fix them all at the same time.

Quoted 10 lines
Quoted 5 lines

We've had the ability to check the git config for quite some time, we could take it further an inhibit the use of the host config if we are worried about breakage like this.

Maybe this would be the best idea overall for our git support inside the build system, avoiding the build user global git config. That probably solves the issue we generally have. I'm okay either way.

Agreed.

There's some complexity with git versions and the variables that are available to inhibit local/host/system configuration, but really, if we want to have isolated and reproducible builds, we should provide our own configuration.

Richard: Does that sound reasonable to you ? If you want to fix this immediate problem, i don't see any harm in the patch, but it may just mask things and ensure we don't ever fix it fully.

William: could you raise a bugzilla (and assign it to me) about git configuration from the user/system influencing our build ? That would ensure we don't forget it completely :)

Bruce

Quoted 26 lines
Quoted 26 lines

Bruce

Quoted 23 lines
---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -2
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
                git init
                check_git_config
                git add .
-               git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+               git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
                git clean -d -f
        fi
 
--

2.42.0.820.g83a721a137-goog

Quoted 14 lines
Quoted 10 lines
Quoted 8 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189694): https://lists.openembedded.org/g/openembedded-core/message/189694
Mute This Topic: https://lists.openembedded.org/mt/102189231/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#4 lore raw
Quoted 59 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Bruce Ashfield
Sent: den 26 oktober 2023 16:26
To: William Kennington <[email protected]>; Richard Purdie <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH] kernel: Commit without running hooks

Quoted 2 lines · William Kennington

On Wed, Oct 25, 2023 at 7:08 PM Bruce Ashfield

Quoted 5 lines · <[email protected]> wrote:
Quoted 5 lines
Quoted 2 lines · William A. Kennington III via lists.openembedded.org

The hooks are pulled from the impure environment and are often

broken in

Quoted 1 line
Quoted 1 line
Quoted 1 line

our environments. There is no reason to add extra metadata or verify

the

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

commit message as its arbitrary to turn the tarball into a git repo.

But what about the other uses of git ? If the hooks are broken during the creation step, other uses of git should also be broken.

The only time hooks are run is for git commit. AFAIK our build process has no other invocations of git commit. All other git operations work fine.

Right. There are a few others that you just aren't hitting. There's the git option for patching (for any recipe), and also some use in the kern-tools that you wouldn't see unless you are also patching or doing merges as part of the kernel patch phase.

It would be nice to fix them all at the same time.

Quoted 10 lines
Quoted 5 lines

We've had the ability to check the git config for quite some time, we could take it further an inhibit the use of the host config if we are worried about breakage like this.

Maybe this would be the best idea overall for our git support inside the build system, avoiding the build user global git config. That probably solves the issue we generally have. I'm okay either way.

Agreed.

There's some complexity with git versions and the variables that are available to inhibit local/host/system configuration, but really, if we want to have isolated and reproducible builds, we should provide our own configuration.

Richard: Does that sound reasonable to you ? If you want to fix this immediate problem, i don't see any harm in the patch, but it may just mask things and ensure we don't ever fix it fully.

Quoted 5 lines

William: could you raise a bugzilla (and assign it to me) about git configuration from the user/system influencing our build ? That would ensure we don't forget it completely :)

Bruce

I would argue that it can be a bad idea to ignore the user's Git configuration. It may, e.g., contain insteadOf settings, which if ignored, prevents the build from fetching from the correct sources. There may be proxy settings that are needed. There may be hooks that are needed. And the list goes on. Trying to avoid problems by completely ignoring the user's configuration is likely to open up another can of problems.

That said, specifically ignoring the pre-commit and commit-msg hooks by specifying -n (or --no-verify as I prefer) to git commit is probably a good idea. At least if others are like us and have global Git hooks that verify that the commit message follows our rules...

//Peter

Quoted 32 lines
Quoted 22 lines
Quoted 22 lines

Bruce

Quoted 19 lines
---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -1
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
                git init
                check_git_config
                git add .
-               git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+               git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
                git clean -d -f
        fi
 
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

--
2.42.0.820.g83a721a137-goog

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#5 lore raw
Quoted 66 lines · Peter Kjellerstedt
Quoted 60 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Bruce Ashfield
Sent: den 26 oktober 2023 16:26
To: William Kennington <[email protected]>; Richard Purdie <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH] kernel: Commit without running hooks

Quoted 2 lines · William Kennington

On Wed, Oct 25, 2023 at 7:08 PM Bruce Ashfield

Quoted 5 lines · <[email protected]> wrote:
Quoted 5 lines
Quoted 2 lines · William A. Kennington III via lists.openembedded.org

The hooks are pulled from the impure environment and are often

broken in

Quoted 1 line
Quoted 1 line
Quoted 1 line

our environments. There is no reason to add extra metadata or verify

the

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

commit message as its arbitrary to turn the tarball into a git repo.

But what about the other uses of git ? If the hooks are broken during the creation step, other uses of git should also be broken.

The only time hooks are run is for git commit. AFAIK our build process has no other invocations of git commit. All other git operations work fine.

Right. There are a few others that you just aren't hitting. There's the git option for patching (for any recipe), and also some use in the kern-tools that you wouldn't see unless you are also patching or doing merges as part of the kernel patch phase.

It would be nice to fix them all at the same time.

Quoted 10 lines
Quoted 5 lines

We've had the ability to check the git config for quite some time, we could take it further an inhibit the use of the host config if we are worried about breakage like this.

Maybe this would be the best idea overall for our git support inside the build system, avoiding the build user global git config. That probably solves the issue we generally have. I'm okay either way.

Agreed.

There's some complexity with git versions and the variables that are available to inhibit local/host/system configuration, but really, if we want to have isolated and reproducible builds, we should provide our own configuration.

Richard: Does that sound reasonable to you ? If you want to fix this immediate problem, i don't see any harm in the patch, but it may just mask things and ensure we don't ever fix it fully.

Quoted 5 lines

William: could you raise a bugzilla (and assign it to me) about git configuration from the user/system influencing our build ? That would ensure we don't forget it completely :)

Bruce

Quoted 8 lines

I would argue that it can be a bad idea to ignore the user's Git configuration. It may, e.g., contain insteadOf settings, which if ignored, prevents the build from fetching from the correct sources. There may be proxy settings that are needed. There may be hooks that are needed. And the list goes on. Trying to avoid problems by completely ignoring the user's configuration is likely to open up another can of problems.

I'd argue that anything critical to the build should be explicitly configured and part of our environment. We don't really want our fetchers to pull sources using locally configured values, why would we let git ?

Relying on a builder or an organization putting these settings in a git configuration that is part of the host is error prone as well.

I'd much rather flush out dependencies like this and have error reports, than to keep them hidden. I'll definitely be continuing to work on inhibiting the host environment and see what pops up.

Quoted 6 lines

That said, specifically ignoring the pre-commit and commit-msg hooks by specifying -n (or --no-verify as I prefer) to git commit is probably a good idea. At least if others are like us and have global Git hooks that verify that the commit message follows our rules...

I only see this as a fallback solution.

Bruce

Quoted 35 lines

//Peter

Quoted 32 lines
Quoted 22 lines
Quoted 22 lines

Bruce

Quoted 19 lines
---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -1
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
                git init
                check_git_config
                git add .
-               git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+               git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
                git clean -d -f
        fi
 
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

--
2.42.0.820.g83a721a137-goog

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw
Quoted 83 lines · Bruce Ashfield via lists.openembedded.org
Quoted 66 lines · Peter Kjellerstedt
Quoted 60 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Bruce Ashfield
Sent: den 26 oktober 2023 16:26
To: William Kennington <[email protected]>; Richard Purdie <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH] kernel: Commit without running hooks

Quoted 2 lines · William Kennington

On Wed, Oct 25, 2023 at 7:08 PM Bruce Ashfield

Quoted 5 lines · <[email protected]> wrote:
Quoted 5 lines
Quoted 2 lines · William A. Kennington III via lists.openembedded.org

The hooks are pulled from the impure environment and are often

broken in

Quoted 1 line
Quoted 1 line
Quoted 1 line

our environments. There is no reason to add extra metadata or verify

the

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

commit message as its arbitrary to turn the tarball into a git repo.

But what about the other uses of git ? If the hooks are broken during the creation step, other uses of git should also be broken.

The only time hooks are run is for git commit. AFAIK our build process has no other invocations of git commit. All other git operations work fine.

Right. There are a few others that you just aren't hitting. There's the git option for patching (for any recipe), and also some use in the kern-tools that you wouldn't see unless you are also patching or doing merges as part of the kernel patch phase.

It would be nice to fix them all at the same time.

Quoted 10 lines
Quoted 5 lines

We've had the ability to check the git config for quite some time, we could take it further an inhibit the use of the host config if we are worried about breakage like this.

Maybe this would be the best idea overall for our git support inside the build system, avoiding the build user global git config. That probably solves the issue we generally have. I'm okay either way.

Agreed.

There's some complexity with git versions and the variables that are available to inhibit local/host/system configuration, but really, if we want to have isolated and reproducible builds, we should provide our own configuration.

Richard: Does that sound reasonable to you ? If you want to fix this immediate problem, i don't see any harm in the patch, but it may just mask things and ensure we don't ever fix it fully.

Quoted 5 lines

William: could you raise a bugzilla (and assign it to me) about git configuration from the user/system influencing our build ? That would ensure we don't forget it completely :)

Bruce

Quoted 8 lines

I would argue that it can be a bad idea to ignore the user's Git configuration. It may, e.g., contain insteadOf settings, which if ignored, prevents the build from fetching from the correct sources. There may be proxy settings that are needed. There may be hooks that are needed. And the list goes on. Trying to avoid problems by completely ignoring the user's configuration is likely to open up another can of problems.

I'd argue that anything critical to the build should be explicitly configured and part of our environment. We don't really want our fetchers to pull sources using locally configured values, why would we let git ?

Quoted 8 lines

Relying on a builder or an organization putting these settings in a git configuration that is part of the host is error prone as well.

I'd much rather flush out dependencies like this and have error reports, than to keep them hidden. I'll definitely be continuing to work on inhibiting the host environment and see what pops up.

I should also add (for the archive's benefit), that I'm only talking about a limited set of places where we'd opt into this isolation (for now). In particular for the kernel-yocto bits, and possibly for the patch.py portions. So just patching / applying changes, I'm not going to muck with devtool or workflows like that.

What they are presumably applying has already had a basic level of sanity applied.

Bruce

Quoted 62 lines
Quoted 7 lines

That said, specifically ignoring the pre-commit and commit-msg hooks by specifying -n (or --no-verify as I prefer) to git commit is probably a good idea. At least if others are like us and have global Git hooks that verify that the commit message follows our rules...

I only see this as a fallback solution.

Bruce

Quoted 35 lines

//Peter

Quoted 32 lines
Quoted 22 lines
Quoted 22 lines

Bruce

Quoted 19 lines
---
 meta/classes-recipe/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel-yocto.bbclass +1 -1
index 4ac977b122..cb9cd26b09 100644
@@ -408,7 +408,7 @@ do_kernel_checkout() {
                git init
                check_git_config
                git add .
-               git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
+               git commit -q -n -m "baseline commit: creating repo for ${PN}-${PV}"
                git clean -d -f
        fi
 
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

--
2.42.0.820.g83a721a137-goog

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189761): https://lists.openembedded.org/g/openembedded-core/message/189761
Mute This Topic: https://lists.openembedded.org/mt/102189231/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#8 lore raw
In reply to William A. Kennington III

Hello William,

On Wed, 25 Oct 2023 15:37:10 -0700
"William A. Kennington III via lists.openembedded.org" <[email protected]> wrote:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

As you can see your sender address has been mangled, and as a result the patch is rejected by the the openembedded git server. This is not your fault, but we need you to modify your git configuration to prevent this from happening in the future. Have a look at the Contributor Guide for more info:

https://docs.yoctoproject.org/contributor-guide/submit-changes.html#fixing-your-from-identity

I'm taking your patch for testing on the autobuilders, fixing it manually so you don't need to resend your patch this time.

Best regards,
Luca

Signature · 3 lines
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#9 lore raw
Quoted 8 lines · Luca Ceresoli

Hello William,

On Wed, 25 Oct 2023 15:37:10 -0700
"William A. Kennington III via lists.openembedded.org" <[email protected]> wrote:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 7 lines

As you can see your sender address has been mangled, and as a result the patch is rejected by the the openembedded git server. This is not your fault, but we need you to modify your git configuration to prevent this from happening in the future. Have a look at the Contributor Guide for more info:

https://docs.yoctoproject.org/contributor-guide/submit-changes.html#fixing-your-from-identity

Ah interesting, I didn't know there was a missing setting and assumed it was our special mail server config that always caused the issue.

Quoted 11 lines

I'm taking your patch for testing on the autobuilders, fixing it manually so you don't need to resend your patch this time.

Best regards,
Luca

--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com