Skip to contentSkip to conversation

patman: Check commit_match before stripping leading whitespace

PATCH New Scott Wood 2 participants 7 messages Sep 25, 2014 – Oct 10, 2014

lore
Share & tools
lore

Scott Wood

author
#0 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -1
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

Hi Scott,

Quoted 36 lines · Scott Wood

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -2
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
--
Quoted 2 lines

1.9.1

Thanks for finding this bug.

This could use a test in tools/patman/test.py

The problem is that you are breaking the patch-processing part of this code. It operates in two modes - see the comment at the top of ProcessLine(). With your change it will not process patches correctly, e.g. to add Commit-notes: to the patch.

Regards,
Simon

author
#2 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

Quoted 44 lines · Simon Glass

Hi Scott,

Quoted 36 lines · Scott Wood

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -2
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
--
Quoted 2 lines

1.9.1

Thanks for finding this bug.

Quoted 6 lines

This could use a test in tools/patman/test.py

The problem is that you are breaking the patch-processing part of this code. It operates in two modes - see the comment at the top of ProcessLine(). With your change it will not process patches correctly, e.g. to add Commit-notes: to the patch.

How would this patch would make any difference in the "self.is_log == False" case? The whitespace removal that this patch reorders around won't be executed, and commit_match will be None regardless.

-Scott

#3 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

Hi Scott,

Quoted 47 lines · Scott Wood
Quoted 46 lines · Simon Glass

Hi Scott,

Quoted 36 lines · Scott Wood

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -2
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
--
Quoted 2 lines

1.9.1

Thanks for finding this bug.

This could use a test in tools/patman/test.py

Quoted 8 lines
Quoted 4 lines

The problem is that you are breaking the patch-processing part of this code. It operates in two modes - see the comment at the top of ProcessLine(). With your change it will not process patches correctly, e.g. to add Commit-notes: to the patch.

How would this patch would make any difference in the "self.is_log == False" case? The whitespace removal that this patch reorders around won't be executed, and commit_match will be None regardless.

You are changing it so that commit_match will always be None for patches.

This means that the commit message in a patch will never be found, so the state machine that tracks where it is in the patch will not function.

Try adding something like:

Commit-notes:
this is a note
more stuff
END

to a commit and then run patman. You will see that the commit notes are not parsed.

Regards,
Simon

author
#4 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

Quoted 55 lines · Simon Glass

Hi Scott,

Quoted 52 lines · Scott Wood
Quoted 50 lines · Simon Glass

Hi Scott,

Quoted 36 lines · Scott Wood

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -2
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
--
Quoted 2 lines

1.9.1

Thanks for finding this bug.

This could use a test in tools/patman/test.py

The problem is that you are breaking the patch-processing part of this code. It operates in two modes - see the comment at the top of ProcessLine(). With your change it will not process patches correctly, e.g. to add Commit-notes: to the patch.

Quoted 5 lines
Quoted 3 lines

How would this patch would make any difference in the "self.is_log == False" case? The whitespace removal that this patch reorders around won't be executed, and commit_match will be None regardless.

You are changing it so that commit_match will always be None for patches.

If by "for patches" you mean when self.is_log == False, then it was always None.

Quoted 10 lines

This means that the commit message in a patch will never be found, so the state machine that tracks where it is in the patch will not function.

Try adding something like:

Commit-notes:
this is a note
more stuff
END

Commit-notes is detected by commit_tag_match, not commit_match.

Quoted 2 lines

to a commit and then run patman. You will see that the commit notes are not parsed.

I just tried it and the commit notes work fine.

BTW, tools/patman/README says "For most cases of using patman for U-Boot development, patman will locate and use the file 'doc/git-mailrc' in your U-Boot directory. This contains most of the aliases you will need." But, this did not work -- patman threw an exception at me saying alias not found -- until I ran "git config sendemail.aliasesfile doc/git-mailrc" which the README doesn't mention.

-Scott

#5 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

Hi Scott,

Quoted 76 lines · Scott Wood
Quoted 55 lines · Simon Glass

Hi Scott,

Quoted 52 lines · Scott Wood
Quoted 50 lines · Simon Glass

Hi Scott,

Quoted 36 lines · Scott Wood

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -2
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
--
Quoted 2 lines

1.9.1

Thanks for finding this bug.

This could use a test in tools/patman/test.py

The problem is that you are breaking the patch-processing part of this code. It operates in two modes - see the comment at the top of ProcessLine(). With your change it will not process patches correctly, e.g. to add Commit-notes: to the patch.

Quoted 5 lines
Quoted 3 lines

How would this patch would make any difference in the "self.is_log == False" case? The whitespace removal that this patch reorders around won't be executed, and commit_match will be None regardless.

You are changing it so that commit_match will always be None for patches.

If by "for patches" you mean when self.is_log == False, then it was always None.

Quoted 10 lines

This means that the commit message in a patch will never be found, so the state machine that tracks where it is in the patch will not function.

Try adding something like:

Commit-notes:
this is a note
more stuff
END

Quoted 6 lines

Commit-notes is detected by commit_tag_match, not commit_match.

Quoted 2 lines

to a commit and then run patman. You will see that the commit notes are not parsed.

I just tried it and the commit notes work fine.

Sorry you are right, I must have typed it incorrectly when I tried it. I just tried it again and it is fine.

Quoted 7 lines

BTW, tools/patman/README says "For most cases of using patman for U-Boot development, patman will locate and use the file 'doc/git-mailrc' in your U-Boot directory. This contains most of the aliases you will need." But, this did not work -- patman threw an exception at me saying alias not found -- until I ran "git config sendemail.aliasesfile doc/git-mailrc" which the README doesn't mention.

OK thanks the report, I sent a patch.

Regards,
Simon

#6 lore raw

PATCH 1/1 patman: Check commit_match before stripping leading whitespace

Quoted 84 lines · Simon Glass

Hi Scott,

Quoted 81 lines · Scott Wood
Quoted 55 lines · Simon Glass

Hi Scott,

Quoted 52 lines · Scott Wood
Quoted 50 lines · Simon Glass

Hi Scott,

Quoted 36 lines · Scott Wood

True commit lines start at column zero. Anything that is indented is part of the commit message instead. I noticed this by trying to run buildman with commit e3a4facdfc07179ebe017a07b8de6224a935a9f3 as master, which contained a reference to a Linux commit inside the commit message. ProcessLine saw that as a genuite commit line, and thus buildman tried to build it, and died with an exception because that SHA is not present in the U-Boot tree.

---
 tools/patman/patchstream.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/patman/patchstream.py +3 -2
index d630157..68e98b9 100644
@@ -139,6 +139,9 @@ class PatchStream:
         # Initially we have no output. Prepare the input line string
         out = []
         line = line.rstrip('\n')
+
+        commit_match = re_commit.match(line) if self.is_log else None
+
         if self.is_log:
             if line[:4] == '    ':
                 line = line[4:]
@@ -146,7 +149,6 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
-        commit_match = re_commit.match(line) if self.is_log else None
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
--
Quoted 2 lines

1.9.1

Thanks for finding this bug.

This could use a test in tools/patman/test.py

The problem is that you are breaking the patch-processing part of this code. It operates in two modes - see the comment at the top of ProcessLine(). With your change it will not process patches correctly, e.g. to add Commit-notes: to the patch.

Quoted 5 lines
Quoted 3 lines

How would this patch would make any difference in the "self.is_log == False" case? The whitespace removal that this patch reorders around won't be executed, and commit_match will be None regardless.

You are changing it so that commit_match will always be None for patches.

If by "for patches" you mean when self.is_log == False, then it was always None.

Quoted 10 lines

This means that the commit message in a patch will never be found, so the state machine that tracks where it is in the patch will not function.

Try adding something like:

Commit-notes:
this is a note
more stuff
END

Commit-notes is detected by commit_tag_match, not commit_match.

Quoted 2 lines

to a commit and then run patman. You will see that the commit notes are not parsed.

Quoted 6 lines
Quoted 1 line

I just tried it and the commit notes work fine.

Sorry you are right, I must have typed it incorrectly when I tried it. I just tried it again and it is fine.

Applied to u-boot-x86/patman, thanks!

Quoted 13 lines
Quoted 8 lines

BTW, tools/patman/README says "For most cases of using patman for U-Boot development, patman will locate and use the file 'doc/git-mailrc' in your U-Boot directory. This contains most of the aliases you will need." But, this did not work -- patman threw an exception at me saying alias not found -- until I ran "git config sendemail.aliasesfile doc/git-mailrc" which the README doesn't mention.

OK thanks the report, I sent a patch.

Regards,
Simon