Skip to contentSkip to conversation

Improve handoff prepare on SoCFPGA

PATCH Superseded Brian Sune 3 participants 7 messages Apr 21 – Apr 25 1 message not in the archive

Subsystems (3)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Brian Sune

author
#0 lore raw

PATCH v4 1/1 Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

As such this ensure qts folder h files are properly updated.

---
 arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)
arch/arm/mach-socfpga/config.mk +24 -3
index 1ca1d33cb16..010ea633f12 100644
@@ -43,6 +43,27 @@ socfpga_g5_handoff_prepare:
 			exit 0; \
 		fi; \
 		echo "[INFO] Found hiof file: $$HIOF_FILE"; \
-		echo "[INFO] Running BSP generator..."; \
-		python3 $(srctree)/tools/cv_bsp_generator/cv_bsp_generator.py -i "$$HANDOFF_PATH" -o "$$BOARD_DIR/qts" || echo "[WARN] BSP generator failed, continuing..."; \
-		echo "[DONE] SoCFPGA QTS handoff conversion complete."
+		echo "[INFO] Try BSP generator..."; \
+		TEMP_DIR=$$(mktemp -dp "$$BOARD_DIR/"); \
+		trap 'rm -rf "$$TEMP_DIR"' EXIT; \
+		if python3 $(srctree)/tools/cv_bsp_generator/cv_bsp_generator.py -i "$$HANDOFF_PATH" -o "$$TEMP_DIR"; then \
+			if [ -n "$${HANDOFF_KEEP+x}" ] && [ "$${HANDOFF_KEEP:-1}" != "0" ]; then \
+				echo "[INFO] Preserving old BSP files..."; \
+				TIMESTAMP=$$(date +%Y%m%d_%H%M%S); \
+				for f in "$$BOARD_DIR"/qts/*.h; do \
+					[ -e "$$f" ] || continue; \
+					echo "[INFO] $$f -> $${f%.h}.h.handoff_backup.$$TIMESTAMP"; \
+					mv "$$f" "$${f%.h}.h.handoff_backup.$$TIMESTAMP"; \
+				done; \
+			else \
+				echo "[INFO] Clean old BSP files..."; \
+				if ls "$$BOARD_DIR/qts"/*.h >/dev/null 2>&1; then \
+					rm "$$BOARD_DIR/qts"/*.h; \
+					echo "[INFO] Removed old BSP files..."; \
+				fi; \
+			fi; \
+			mv "$$TEMP_DIR"/*.h "$$BOARD_DIR"/qts; \
+			echo "[INFO] SoCFPGA QTS handoff conversion complete."; \
+		else \
+			echo "[WARN] BSP generator failed!"; \
+		fi;
Signature · 1 line
2.34.1
Reviewed-by on v4 1/1
#1 lore raw

Hi Brian,

Quoted 14 lines · Sune Brian

Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

Quoted 7 lines

As such this ensure qts folder h files are properly updated.

arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---

 1 file changed, 24 insertions(+), 3 deletions(-)

Some thoughts though.

We try to add a change log for each revision and use imperative present tense (Introduce a temp folder... rather than This patch introduces...). Spell out Python script rather than PY script, and reword as prose or a bullet list instead of 1), 2), ... You could also explain *why* the qts files were sometimes not being replaced - the motivation paragraph currently just says some cases without saying what goes wrong.

The message says kept files are renamed to xxx.h.handoff.<timestamp> but the code uses .h.handoff_backup.<timestamp> - they should match.

arch/arm/mach-socfpga/config.mk socfpga_g5_handoff_prepare: · line 43

Quoted 3 lines
arch/arm/mach-socfpga/config.mk +1 -0
@@ -43,6 +43,27 @@ socfpga_g5_handoff_prepare:
+                     if [ -n '$${HANDOFF_KEEP+x}' ] && [ '$${HANDOFF_KEEP:-1}' != '0' ]; then \

How about:

if [ '$${HANDOFF_KEEP:-0}' != '0' ] ; then \

Regards,
Simon

author
#2 lore raw
Quoted 21 lines · Simon Glass

Hi Brian,

Quoted 17 lines · Sune Brian

Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

As such this ensure qts folder h files are properly updated.

Quoted 9 lines
Quoted 4 lines

arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---

 1 file changed, 24 insertions(+), 3 deletions(-)

Some thoughts though.

Hi Simon.

Quoted 5 lines

We try to add a change log for each revision and use imperative present tense (Introduce a temp folder... rather than This patch introduces...). Spell out Python script rather than PY script, and reword as prose or a bullet list instead of 1), 2), ... You could also explain *why* the qts files were sometimes not being replaced - the

Because it is not fully traceable, git pull sometimes could introduce some kind of unknown cases that even run make prepare will not update the h files or the python script itself does not fully replace it from first place.
So the easiest way to play safe in any possible condition is [remove and replace] or simply [rename and create].

And I don't even fully confirm the cases, while this patch does improve the previous .mk operation no matter if it is solving the unreplacing issue or not.

So the motivation was explained from the title itself.

Quoted 5 lines

motivation paragraph currently just says some cases without saying what goes wrong.

The message says kept files are renamed to xxx.h.handoff.<timestamp> but the code uses .h.handoff_backup.<timestamp> - they should match.

That my mistake on the description, indeed it should be .h.handoff_backup.<timestamp>

The idea just follows T.F. previous optional idea so I don't fully pay attention to the wordings.

Quoted 7 lines

arch/arm/mach-socfpga/config.mk socfpga_g5_handoff_prepare: · line 43

Quoted 4 lines
arch/arm/mach-socfpga/config.mk +1 -0
@@ -43,6 +43,27 @@ socfpga_g5_handoff_prepare:
+                     if [ -n '$${HANDOFF_KEEP+x}' ] && [ '$${HANDOFF_KEEP:-1}' != '0' ]; then \

How about:

No I don't like this because you ignored my original idea on:
1) HANDOFF_KEEP= should run keep
because HANDOFF_KEEP cannot without = in .mk usage
2) When users use this flag, hence, it is telling that they want to keep the thingy.
So it makes no sense to =1 or =0 to further control it from first place. This flag is inherently boolean logic.
But due to your suggestion I further elaborated this boolean flag to support =0 or =1 or =x cases.

Sorry for this part, I will not change.

Cheers,
Brian

Quoted 4 lines

if [ '$${HANDOFF_KEEP:-0}' != '0' ] ; then \

Regards,
Simon

Reviewed-by
#3 lore raw

Hi Brian,

Quoted 28 lines · Sune Brian

[CAUTION: This email is from outside your organization. Unless you trust the sender, do not click on links or open attachments as it may be a fraudulent email attempting to steal your information and/or compromise your computer.]

Quoted 25 lines · Simon Glass

Hi Brian,

Quoted 14 lines · Sune Brian

Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

Quoted 7 lines

As such this ensure qts folder h files are properly updated.

arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---

  1 file changed, 24 insertions(+), 3 deletions(-)

Please roll a v5 with:
1) Commit message: imperative tense, add a short v5 changelog vs v4, spell out "Python", and fix the backup filename in the text to match the code (.h.handoff_backup.<timestamp>).

2) HANDOFF_KEEP: I am fine keeping your current semantics [OR: please adopt Simon's single test ...]. Either way, add a brief comment in config.mk next to the recipe stating: unset => remove old headers; HANDOFF_KEEP set and not 0 => rename old *.h to *.h.handoff_backup.*; HANDOFF_KEEP=0 => remove old headers even when the variable is set.

3) [Optional] If HANDOFF_KEEP= (empty) should not mean "keep", adjust the shell logic or document the current behavior.

With that I can apply from list.

Thanks.

Best regards,

Tien Fong

#4 lore raw
In reply to Sune Brian

Hi Brian,

Quoted 84 lines · Sune Brian
Quoted 21 lines · Simon Glass

Hi Brian,

Quoted 17 lines · Sune Brian

Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

As such this ensure qts folder h files are properly updated.

Quoted 9 lines
Quoted 4 lines

arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---

 1 file changed, 24 insertions(+), 3 deletions(-)

Some thoughts though.

Hi Simon.

Quoted 5 lines

We try to add a change log for each revision and use imperative present tense (Introduce a temp folder... rather than This patch introduces...). Spell out Python script rather than PY script, and reword as prose or a bullet list instead of 1), 2), ... You could also explain *why* the qts files were sometimes not being replaced - the

Because it is not fully traceable, git pull sometimes could introduce some kind of unknown cases that even run make prepare will not update the h files or the python script itself does not fully replace it from first place.
So the easiest way to play safe in any possible condition is [remove and replace] or simply [rename and create].

And I don't even fully confirm the cases, while this patch does improve the previous .mk operation no matter if it is solving the unreplacing issue or not.

So the motivation was explained from the title itself.

Quoted 5 lines

motivation paragraph currently just says some cases without saying what goes wrong.

The message says kept files are renamed to xxx.h.handoff.<timestamp> but the code uses .h.handoff_backup.<timestamp> - they should match.

That my mistake on the description, indeed it should be .h.handoff_backup.<timestamp>

The idea just follows T.F. previous optional idea so I don't fully pay attention to the wordings.

Quoted 7 lines

arch/arm/mach-socfpga/config.mk socfpga_g5_handoff_prepare: · line 43

Quoted 4 lines
arch/arm/mach-socfpga/config.mk +1 -0
@@ -43,6 +43,27 @@ socfpga_g5_handoff_prepare:
+                     if [ -n '$${HANDOFF_KEEP+x}' ] && [ '$${HANDOFF_KEEP:-1}' != '0' ]; then \

How about:

No I don't like this because you ignored my original idea on:
1) HANDOFF_KEEP= should run keep
because HANDOFF_KEEP cannot without = in .mk usage
2) When users use this flag, hence, it is telling that they want to keep the thingy.
So it makes no sense to =1 or =0 to further control it from first place. This flag is inherently boolean logic.
But due to your suggestion I further elaborated this boolean flag to support =0 or =1 or =x cases.

Quoted 1 line

Sorry for this part, I will not change.

That's fine...note that you have my review tag, so you can add that to a new version, if you do one.

Quoted 5 lines

Cheers,
Brian

Quoted 1 line

if [ '$${HANDOFF_KEEP:-0}' != '0' ] ; then \

Regards,
Simon

author
#5 lore raw
Quoted 88 lines · Simon Glass

Hi Brian,

Quoted 84 lines · Sune Brian
Quoted 21 lines · Simon Glass

Hi Brian,

Quoted 17 lines · Sune Brian

Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

As such this ensure qts folder h files are properly updated.

Quoted 9 lines
Quoted 4 lines

arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---

 1 file changed, 24 insertions(+), 3 deletions(-)

Some thoughts though.

Hi Simon.

Quoted 5 lines

We try to add a change log for each revision and use imperative present tense (Introduce a temp folder... rather than This patch introduces...). Spell out Python script rather than PY script, and reword as prose or a bullet list instead of 1), 2), ... You could also explain *why* the qts files were sometimes not being replaced - the

Because it is not fully traceable, git pull sometimes could introduce some kind of unknown cases that even run make prepare will not update the h files or the python script itself does not fully replace it from first place.
So the easiest way to play safe in any possible condition is [remove and replace] or simply [rename and create].

And I don't even fully confirm the cases, while this patch does improve the previous .mk operation no matter if it is solving the unreplacing issue or not.

So the motivation was explained from the title itself.

Quoted 5 lines

motivation paragraph currently just says some cases without saying what goes wrong.

The message says kept files are renamed to xxx.h.handoff.<timestamp> but the code uses .h.handoff_backup.<timestamp> - they should match.

That my mistake on the description, indeed it should be .h.handoff_backup.<timestamp>

The idea just follows T.F. previous optional idea so I don't fully pay attention to the wordings.

Quoted 7 lines

arch/arm/mach-socfpga/config.mk socfpga_g5_handoff_prepare: · line 43

Quoted 4 lines
arch/arm/mach-socfpga/config.mk +1 -0
@@ -43,6 +43,27 @@ socfpga_g5_handoff_prepare:
+                     if [ -n '$${HANDOFF_KEEP+x}' ] && [ '$${HANDOFF_KEEP:-1}' != '0' ]; then \

How about:

No I don't like this because you ignored my original idea on:
1) HANDOFF_KEEP= should run keep
because HANDOFF_KEEP cannot without = in .mk usage
2) When users use this flag, hence, it is telling that they want to keep the thingy.
So it makes no sense to =1 or =0 to further control it from first place. This flag is inherently boolean logic.
But due to your suggestion I further elaborated this boolean flag to support =0 or =1 or =x cases.

Quoted 10 lines
Quoted 1 line

Sorry for this part, I will not change.

That's fine...note that you have my review tag, so you can add that to a new version, if you do one.

Quoted 5 lines

Cheers,
Brian

Quoted 1 line

if [ '$${HANDOFF_KEEP:-0}' != '0' ] ; then \

Hi Simon,

V5 has been released.
Due to T.F. "please to use your check condition"

Logically speaking.

The check condition now has become:

A person has no intention to use the restroom.
But that person intentionally walk in front of the restroom door and speak out not going to use the restroom,
Then that person simply goes back to where he came from and does not even enter the restroom.

What a wonderful world =]

Enjoy,
Brian

Quoted 2 lines

Regards,
Simon

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#7 lore raw
Quoted 111 lines · Tien Fong

Hi Brian,

Quoted 98 lines · Sune Brian

[CAUTION: This email is from outside your organization. Unless you trust the sender, do not click on links or open attachments as it may be a fraudulent email attempting to steal your information and/or compromise your computer.]

Quoted 78 lines · Simon Glass

Hi Brian,

Quoted 75 lines · Sune Brian
Quoted 20 lines · Simon Glass

Hi Brian,

Quoted 17 lines · Sune Brian

Improve handoff prepare on SoCFPGA

There are some cases that the Python scripts are run and the qts files are not replaced.

This patch introduces:
1) TMP folder for Python script
2) TMP folder will automatically remove on exit
3) If PY script fails, the replace will not conduct.
4) Only when PY script passed the replace with keep or without keep will be done via NEW HANDOFF_KEEP=xxx
5) When keep is selected the old files are renamed to xxx.h.handoff.<timestamp>

As such this ensure qts folder h files are properly updated.

Quoted 8 lines
Quoted 4 lines

arch/arm/mach-socfpga/config.mk | 27 ++++++++++++++++++++++++---

  1 file changed, 24 insertions(+), 3 deletions(-)

Some thoughts though.

Hi Simon.

Quoted 5 lines

We try to add a change log for each revision and use imperative present tense (Introduce a temp folder... rather than This patch introduces...). Spell out Python script rather than PY script, and reword as prose or a bullet list instead of 1), 2), ... You could also explain *why* the qts files were sometimes not being replaced - the

Because it is not fully traceable, git pull sometimes could introduce some kind of unknown cases that even run make prepare will not update the h files or the python script itself does not fully replace it from first place.
So the easiest way to play safe in any possible condition is [remove and replace] or simply [rename and create].

And I don't even fully confirm the cases, while this patch does improve the previous .mk operation no matter if it is solving the unreplacing issue or not.

So the motivation was explained from the title itself.

Quoted 5 lines

motivation paragraph currently just says some cases without saying what goes wrong.

The message says kept files are renamed to xxx.h.handoff.<timestamp> but the code uses .h.handoff_backup.<timestamp> - they should match.

That my mistake on the description, indeed it should be .h.handoff_backup.<timestamp>

The idea just follows T.F. previous optional idea so I don't fully pay attention to the wordings.

Quoted 5 lines

arch/arm/mach-socfpga/config.mk socfpga_g5_handoff_prepare: · line 43

Quoted 3 lines
arch/arm/mach-socfpga/config.mk +1 -0
@@ -43,6 +43,27 @@ socfpga_g5_handoff_prepare:
+                     if [ -n '$${HANDOFF_KEEP+x}' ] && [ '$${HANDOFF_KEEP:-1}' != '0' ]; then \

How about:

No I don't like this because you ignored my original idea on:
1) HANDOFF_KEEP= should run keep
because HANDOFF_KEEP cannot without = in .mk usage
2) When users use this flag, hence, it is telling that they want to keep the thingy.
So it makes no sense to =1 or =0 to further control it from first place. This flag is inherently boolean logic.
But due to your suggestion I further elaborated this boolean flag to support =0 or =1 or =x cases.

Quoted 8 lines
Quoted 1 line

Sorry for this part, I will not change.

That's fine...note that you have my review tag, so you can add that to a new version, if you do one.

Quoted 4 lines

Cheers,
Brian

Quoted 1 line

if [ '$${HANDOFF_KEEP:-0}' != '0' ] ; then \

Hi Simon,

V5 has been released.
Due to T.F. "please to use your check condition"

Logically speaking.

The check condition now has become:

Quoted 7 lines

A person has no intention to use the restroom.
But that person intentionally walk in front of the restroom door and speak out not going to use the restroom,
Then that person simply goes back to where he came from and does not even enter the restroom.

What a wonderful world =]

Quoted 7 lines

Just to clarify my earlier comments, the intent wasn’t to force a particular approach,

but to share review suggestions and ensure the commit message and behavior are clear for future readers.

On the shell test, I had meant: either your earlier semantics or Simon’s

Hi T.F.,

Interesting, "simplified one" so when users use > make + keep flag where flag label is self explained still adding extra =0 or =1 is considered as simplified.

No one forced me, while you used "please" so no worries, just pleasure for me to copy and paste w/o further thinking is even easy for me. Love to do so.

Thanks,
Brian

Quoted 10 lines

simplified one was fine, as long as the behavior is clear for reviewers

I appreciate you incorporating the parts you agreed with.

Best regards,

Tien Fong