Skip to contentSkip to conversation

CMake: Support doc builds on Windows machines that do not have .py file association

PATCH In review Frank Lichtenheld 4 participants 7 messages Jul 4, 2023 – Jul 6, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 CMake: Support doc builds on Windows machines that do not have .py file association

On Windows we might need to call python because .py files are not directly executable. This is true e.g. for GHA runners.
For now we assume that rst2html and rst2man can be handled in the same way and do not test both of them.

Commit e8881ec6dd63bd80ce05202573eac54ab8657fcb unconditionally used $PYTHON, but that broke build on systems where the default python can't be used and we need to respect the shebang. Commit 5dbec1c019d14880ae7bf364b062d3589c7fd9e7 unconditionally did not use $PYTHON, but that broke build on the aformentioned GHA runners.
This commit tries to establish a solution that works for both systems.

---
 doc/CMakeLists.txt | 42 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)
doc/CMakeLists.txt +37 -5
index 2fba80bb..5e729efa 100644
@@ -1,14 +1,46 @@
 set(_GENERATE_HTML_DOC YES)
 set(_GENERATE_MAN_DOC  YES)
+set(_MAYBE_PYTHON "")
 find_program(RST2HTML NAMES rst2html rst2html.py)
 find_program(RST2MAN  NAMES rst2man  rst2man.py)
 
 if (RST2HTML STREQUAL "RST2HTML-NOTFOUND")
     message(STATUS "rst2html not found, not generating HTML documentation")
     set(_GENERATE_HTML_DOC NO)
+else ()
+    # We only run this for RST2HTML and assume the result would be the same
+    # for RST2MAN
+    if (DEFINED CACHE{DOCUTILS_NEED_PYTHON})
+        if ($CACHE{DOCUTILS_NEED_PYTHON})
+            set(_MAYBE_PYTHON ${PYTHON})
+        endif ()
+    else ()
+        execute_process(
+            COMMAND ${RST2HTML} --version
+            OUTPUT_VARIABLE RST2HTML_VERSION_EXE
+            )
+        execute_process(
+            COMMAND ${PYTHON} ${RST2HTML} --version
+            OUTPUT_VARIABLE RST2HTML_VERSION_PY
+            )
+        set(_DOCUTILS_NEED_PYTHON OFF)
+        if(RST2HTML_VERSION_EXE STREQUAL "")
+            if(RST2HTML_VERSION_PY STREQUAL "")
+                message(STATUS "${RST2HTML} found but not working, not generating documentation")
+                set(_GENERATE_HTML_DOC NO)
+                set(_GENERATE_MAN_DOC NO)
+            else ()
+                message(STATUS "${RST2HTML} needs to be executed by ${PYTHON} to work")
+                set(_MAYBE_PYTHON ${PYTHON})
+                set(_DOCUTILS_NEED_PYTHON ON)
+            endif ()
+        endif ()
+        set(DOCUTILS_NEED_PYTHON ${_DOCUTILS_NEED_PYTHON} CACHE BOOL
+            "Whether we need to call python instead of rst2*.py directly")
+    endif (DEFINED CACHE{DOCUTILS_NEED_PYTHON})
 endif ()
 if (RST2MAN STREQUAL "RST2MAN-NOTFOUND")
-    message(STATUS "rst2man not found, not generating HTML documentation")
+    message(STATUS "rst2man not found, not generating man pages")
     set(_GENERATE_MAN_DOC NO)
 endif ()
 
@@ -50,13 +82,13 @@ if (_GENERATE_HTML_DOC)
     list(APPEND ALL_DOCS openvpn.8.html openvpn-examples.5.html)
     add_custom_command(
         OUTPUT openvpn.8.html
-        COMMAND ${RST2HTML} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn.8.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn.8.html
+        COMMAND ${_MAYBE_PYTHON} ${RST2HTML} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn.8.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn.8.html
         MAIN_DEPENDENCY openvpn.8.rst
         DEPENDS ${OPENVPN_SECTIONS}
         )
     add_custom_command(
         OUTPUT openvpn-examples.5.html
-        COMMAND ${RST2HTML} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn-examples.5.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn-examples.5.html
+        COMMAND ${_MAYBE_PYTHON} ${RST2HTML} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn-examples.5.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn-examples.5.html
         MAIN_DEPENDENCY openvpn-examples.5.rst
         DEPENDS ${OPENVPN_EXAMPLES_SECTIONS}
         )
@@ -65,13 +97,13 @@ if (_GENERATE_MAN_DOC)
     list(APPEND ALL_DOCS openvpn.8 openvpn-examples.5)
     add_custom_command(
         OUTPUT openvpn.8
-        COMMAND ${RST2MAN} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn.8.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn.8
+        COMMAND ${_MAYBE_PYTHON} ${RST2MAN} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn.8.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn.8
         MAIN_DEPENDENCY openvpn.8.rst
         DEPENDS ${OPENVPN_SECTIONS}
         )
     add_custom_command(
         OUTPUT openvpn-examples.5
-        COMMAND ${RST2MAN} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn-examples.5.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn-examples.5
+        COMMAND ${_MAYBE_PYTHON} ${RST2MAN} ${RST_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/openvpn-examples.5.rst ${CMAKE_CURRENT_BINARY_DIR}/openvpn-examples.5
         MAIN_DEPENDENCY openvpn-examples.5.rst
         DEPENDS ${OPENVPN_EXAMPLES_SECTIONS}
         )
Signature · 1 line
2.34.1
Acked-by on v1 1/1
#1 lore raw

Am 04.07.23 um 16:09 schrieb Frank Lichtenheld:

Quoted 5 lines

On Windows we might need to call python because .py files are not directly executable. This is true e.g. for GHA runners.
For now we assume that rst2html and rst2man can be handled in the same way and do not test both of them.

Quoted 9 lines

Commit e8881ec6dd63bd80ce05202573eac54ab8657fcb unconditionally used $PYTHON, but that broke build on systems where the default python can't be used and we need to respect the shebang. Commit 5dbec1c019d14880ae7bf364b062d3589c7fd9e7 unconditionally did not use $PYTHON, but that broke build on the aformentioned GHA runners.
This commit tries to establish a solution that works for both systems.

This is annoying and should be necessary but I also don't have no better idea how to handle this better.

#2 lore raw

PATCH 1/1 GHA: Add work-around for rst2*.py not being directly executable on Windows

In reply to Frank Lichtenheld

We write a small .bat file wrapper and then force CMake to use that.

Note that we need to specify the path with / instead of \
as path separator because otherwise run-cmake action will mangle it.

Change-Id I05d3f7f3f7f7418d1977e523c6dcfb6fa5feb604
---
 .github/workflows/build.yaml | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

This is an alternate patch to "CMake: Support doc builds on Windows machines that do not have .py file association" since there was some doubt on IRC whether we want to have the complexity inside of CMake or should delegate that to the caller.

Let me know your opinions.

.github/workflows/build.yaml +20 -1
index da2a427c..72edb089 100644
@@ -286,8 +286,25 @@ jobs:
       - uses: actions/[email protected]...
       - uses: lukka/[email protected]...
 
+      # since we can't execute .py files directly, do a workaround for rst2html,rst2man
+      - name: Determine Python script path
+        run: |
+          import os
+          import sysconfig
+          scripts_path = sysconfig.get_path("scripts",f"{os.name}")
+          with open(os.environ["GITHUB_ENV"], "a") as env:
+            print(f"PYTHON_SCRIPTS_DIR={scripts_path}", file=env)
+        shell: python
       - name: Install rst2html
-        run: python -m pip install --upgrade pip docutils
+        run: |
+          python -m pip install --upgrade pip docutils
+          $install_dir = "${{ runner.workspace }}\docutils"
+          mkdir $install_dir
+          echo "python $Env:PYTHON_SCRIPTS_DIR\rst2html.py %*" > $install_dir\rst2html.bat
+          cat $install_dir\rst2html.bat
+          echo "python $Env:PYTHON_SCRIPTS_DIR\rst2man.py %*" > $install_dir\rst2man.bat
+          $safe_path = $install_dir.replace("\", "/")
+          echo "DOCUTILS=$safe_path" >> "$Env:GITHUB_ENV"
 
       - name: Restore artifacts, or setup vcpkg (do not install any package)
         uses: lukka/[email protected]...
@@ -300,6 +317,7 @@ jobs:
         if: ${{ matrix.arch == 'arm64' }}
         with:
           configurePreset: win-${{ matrix.arch }}-release
+          configurePresetAdditionalArgs: "[`-DRST2HTML=${{ env.DOCUTILS }}/rst2html.bat`, `-DRST2MAN=${{ env.DOCUTILS }}/rst2man.bat`]"
           buildPreset: win-${{ matrix.arch }}-release
 
       - name: Run CMake with vcpkg.json manifest
@@ -307,6 +325,7 @@ jobs:
         if: ${{ matrix.arch != 'arm64' }}
         with:
           configurePreset: win-${{ matrix.arch }}-release
+          configurePresetAdditionalArgs: "[`-DRST2HTML=${{ env.DOCUTILS }}/rst2html.bat`, `-DRST2MAN=${{ env.DOCUTILS }}/rst2man.bat`]"
           buildPreset: win-${{ matrix.arch }}-release
           testPreset: win-${{ matrix.arch }}-release
 
Signature · 1 line
2.34.1
#3 lore raw

Re: [Openvpn-devel] [PATCH] GHA: Add work-around for rst2*.py not being directly executable on Windows

Quoted 13 lines · Frank Lichtenheld

We write a small .bat file wrapper and then force CMake to use that.

Note that we need to specify the path with / instead of \
as path separator because otherwise run-cmake action will mangle it.

Change-Id I05d3f7f3f7f7418d1977e523c6dcfb6fa5feb604
---
 .github/workflows/build.yaml | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
Quoted 6 lines

This is an alternate patch to "CMake: Support doc builds on Windows machines that do not have .py file association" since there was some doubt on IRC whether we want to have the complexity inside of CMake or should delegate that to the caller.

Let me know your opinions.

Personally I prefer the original solution inside CMake. It just solves the problem in a wide range of potential setups without additional effort required by the caller. I think the slightly increased complexity can be accepted for that.

Regards,

Signature · 1 line
  Frank Lichtenheld
#4 lore raw

Re: [Openvpn-devel] [PATCH] GHA: Add work-around for rst2*.py not being directly executable on Windows

Am 06.07.23 um 12:21 schrieb Frank Lichtenheld:

Quoted 19 lines · Frank Lichtenheld
Quoted 18 lines

We write a small .bat file wrapper and then force CMake to use that.

Note that we need to specify the path with / instead of \
as path separator because otherwise run-cmake action will mangle it.

Change-Id I05d3f7f3f7f7418d1977e523c6dcfb6fa5feb604
---
  .github/workflows/build.yaml | 21 ++++++++++++++++++++-
  1 file changed, 20 insertions(+), 1 deletion(-)

This is an alternate patch to "CMake: Support doc builds on Windows machines that do not have .py file association" since there was some doubt on IRC whether we want to have the complexity inside of CMake or should delegate that to the caller.

Quoted 6 lines
Quoted 1 line

Let me know your opinions.

Personally I prefer the original solution inside CMake. It just solves the problem in a wide range of potential setups without additional effort required by the caller. I think the slightly increased complexity can be accepted for that.

I concur for the same reasons. This is also harder for people to replicate on their own setups. They will have to manually write scripts instead of jsut having cmake dealing with the problem.

Arne

#5 lore raw

Re: [Openvpn-devel] [PATCH] GHA: Add work-around for rst2*.py not being directly executable on Windows

I had this problem on both arm64 and x64 machines and CMake patch fixed it. So I'd also go with that.

to 6. heinäk. 2023 klo 13.46 Arne Schwabe ([email protected]...) kirjoitti:

Quoted 40 lines

Am 06.07.23 um 12:21 schrieb Frank Lichtenheld:

Quoted 19 lines · Frank Lichtenheld
Quoted 18 lines

We write a small .bat file wrapper and then force CMake to use that.

Note that we need to specify the path with / instead of \
as path separator because otherwise run-cmake action will mangle it.

Change-Id I05d3f7f3f7f7418d1977e523c6dcfb6fa5feb604
---
  .github/workflows/build.yaml | 21 ++++++++++++++++++++-
  1 file changed, 20 insertions(+), 1 deletion(-)

This is an alternate patch to "CMake: Support doc builds on Windows machines that do not have .py file association" since there was some doubt on IRC whether we want to have the complexity inside of CMake or should delegate that to the caller.

Quoted 6 lines
Quoted 1 line

Let me know your opinions.

Personally I prefer the original solution inside CMake. It just solves the problem in a wide range of potential setups without additional effort required by the caller. I think the slightly increased complexity can be accepted for that.

I concur for the same reasons. This is also harder for people to replicate on their own setups. They will have to manually write scripts instead of jsut having cmake dealing with the problem.

Arne

_______________________________________________
Openvpn-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openvpn-devel

Signature · 1 line
-Lev
#6 lore raw
In reply to Frank Lichtenheld

Your patch has been applied to the master branch.

commit 22213a8834ba5ba5c9818015730edbf3766ad915

Author: Frank Lichtenheld
Date:   Tue Jul 4 15:09:02 2023 +0200
     CMake: Support doc builds on Windows machines that do not have .py file association

     Signed-off-by: Frank Lichtenheld <[email protected]...>
     Acked-by: Arne Schwabe <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg26813.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering