git-p4: fix failed submit by skip non-text data files

From: "dorgon.chang" <[email protected]>

If the submit contain binary files, it will throw exception and stop submit when try to append diff line description.

This commit will skip non-text data files when exception UnicodeDecodeError thrown.

---
git-p4: fix failed submit by skip non-text data files

git-p4: fix failed submit by skip non-text data files

If the submit contain binary files, it will throw exception and stop submit when try to append diff line description.

This commit will skip non-text data files when exception UnicodeDecodeError thrown.

I am using git-p4 with UnrealEngine game projects and this fix works for me.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-977%2Fdorgonman%2Fdorgon%2Ffix_gitp4_get_diff_description-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-977/dorgonman/dorgon/fix_gitp4_get_diff_description-v1
Pull-Request: https://github.com/git/git/pull/977

 git-p4.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

index 4433ca53de7e..29a8c202399a 100755
@@ -1977,8 +1977,11 @@ def get_diff_description(self, editedFiles, filesToAdd, symlinks):
                 newdiff += "+%s\n" % os.readlink(newFile)
             else:
                 f = open(newFile, "r")
-                for line in f.readlines():
-                    newdiff += "+" + line
+                try:
+                    for line in f.readlines():
+                        newdiff += "+" + line
+                except UnicodeDecodeError:
+                    pass # Fond non-text data
                 f.close()
 
         return (diff + newdiff).replace('\r\n', '\n')

base-commit: d4a392452e292ff924e79ec8458611c0f679d6d4

gitgitgadget

As suggested on
https://github.com/git/git/pull/977#issuecomment-862197824, I'm happy to state that the patch looks good to me. IIRC the diff there is solely for the submit template, so it should only include text. That your patch ensures in what seems an idiomatic way.

Simon

Hi Simon,

Thank you for reviewing and chiming in.

The typical way to record your review is to say Reviewed-by:. The Signed-off-by: footer is usually used to indicate that you wrote the patch, or that you shepherd it onto the Git mailing list.

Sorry to be so nit-picky...

Thanks,
Dscho

As suggested on
https://github.com/git/git/pull/977#issuecomment-862197824, I'm happy to state that the patch looks good to me. IIRC the diff there is solely for the submit template, so it should only include text. That your patch ensures in what seems an idiomatic way.

Simon

OK, that explains how the program fails.

If there are changes in aText and aBinary file and you try to submit a cl that contains both changes, you do want changes to both files go together, no? If you skip non-text, does that mean you ignore the changes to aBinary file and submit only the changes to aText file?

I guess my confusion comes from not understanding what you exactly mean by "append diff line description". Whatever that means, if that is purely informational and does not affect what is actually submit in the resulting cl, then the patch would be an improvement. If not, and if for example it loses changes to binary files, then it is merely sweeping the problem under the rug.

In short the explanation of the solution does not build confidence in the readers minds. You'd need to explain why such a skipping is a safe thing to do a bit better.

Even if we assuming that what happens in the loop you threw in try/except block is purely cosmetic and optional thing that does not affect the correct operation of the program or its outcome, I wonder if we can do better. When you get a decode error, you'd have an early part of the change (which could be empty) before you hit the error in newdiff, and that is returned to the caller without any sign that it is a truncated output. I wonder something like

except UnicodeDecodeError:
newdiff = '<<new binary file>>'

may be more helpful to the user. Assuming that this is purely for human consumption without affecting the correctness or outcome of the program and we can place pretty much any text there, that is. But because the proposed commit log message does not explain why skipping is safe, I do not know if that assumption holds in the first place.

Thanks.

The skip will not affect actual submit files in the resulting cl, the diff line description will only appear in submit template, so you can review what changed before actully submit to p4.

I don't know if add any message here will be helpful for users,
so I choose to just skip binary content, since it already append filename previously.

Just fixed the typo, thanks.

These are both good things to write in the proposed log message.

Thanks.

From: "dorgon.chang" <[email protected]>

If the submit contain binary files, it will throw exception and stop submit when try to append diff line description.

This commit will skip non-text data files when exception UnicodeDecodeError thrown.

The skip will not affect actual submit files in the resulting cl, the diff line description will only appear in submit template, so you can review what changed before actully submit to p4.

I don't know if add any message here will be helpful for users,
so I choose to just skip binary content, since it already append filename previously.

---
git-p4: fix failed submit by skip non-text data files

git-p4: fix failed submit by skip non-text data files

If the submit contain binary files, it will throw exception and stop submit when try to append diff line description.

This commit will skip non-text data files when exception UnicodeDecodeError thrown.

I am using git-p4 with UnrealEngine game projects and this fix works for me.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-977%2Fdorgonman%2Fdorgon%2Ffix_gitp4_get_diff_description-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-977/dorgonman/dorgon/fix_gitp4_get_diff_description-v2
Pull-Request: https://github.com/git/git/pull/977

Range-diff vs v1:

1: 19b59f40b183 ! 1: 606729bda112 git-p4: fix failed submit by skip non-text data files
@@ Commit message

This commit will skip non-text data files when exception UnicodeDecodeError thrown.

+ The skip will not affect actual submit files in the resulting cl,
+ the diff line description will only appear in submit template,
+ so you can review what changed before actully submit to p4.
+
+ I don't know if add any message here will be helpful for users,
+ so I choose to just skip binary content, since it already append filename previously.
+

## git-p4.py ##
@@ git-p4.py: def get_diff_description(self, editedFiles, filesToAdd, symlinks):
+ for line in f.readlines():
+ newdiff += "+" + line
+ except UnicodeDecodeError:
-+ pass # Fond non-text data
++ pass # Found non-text data and skip, since diff description should only include text
f.close()

return (diff + newdiff).replace('\r\n', '\n')

 git-p4.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

index 4433ca53de7e..dc1f46351845 100755
@@ -1977,8 +1977,11 @@ def get_diff_description(self, editedFiles, filesToAdd, symlinks):
                 newdiff += "+%s\n" % os.readlink(newFile)
             else:
                 f = open(newFile, "r")
-                for line in f.readlines():
-                    newdiff += "+" + line
+                try:
+                    for line in f.readlines():
+                        newdiff += "+" + line
+                except UnicodeDecodeError:
+                    pass # Found non-text data and skip, since diff description should only include text
                 f.close()
 
         return (diff + newdiff).replace('\r\n', '\n')

base-commit: d4a392452e292ff924e79ec8458611c0f679d6d4

gitgitgadget

This is a crucial piece of information lacking in the proposed commit log message that would help readers understand why this is a safe change. An updated patch with a better log message would be appreciated.

Yes to both.

It is unusual to see "reviewed-by" from those whose names do not appear even once in output of "git shortlog --no-merges git-p4.py" on a patch that touches "git-p4.py", but this is a fringe area (compared to the more core-ish part of the system) where people touch to scratch their own itch without staying around for a long haul, so it is understandable that we do not always have resident experts in the area. A review like this is highly appreciated.

Thanks, all.