Skip to contentSkip to conversation

python3: fix ftplib with TLS 1.3

PATCH In review Ross Burton 3 participants 9 messages Sep 10, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/4 python3: fix ftplib with TLS 1.3

With OpenSSL 1.1.x TLS 1.3 can be used, so backport a patch from Python 3.6 to fix the ftplib unit test.

---
 meta/recipes-devtools/python/python3/ftplib.patch | 57 +++++++++++++++++++++++
 meta/recipes-devtools/python/python3_3.5.5.bb     |  1 +
 2 files changed, 58 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/ftplib.patch
meta/recipes-devtools/python/python3/ftplib.patch new file +57 -0
new file mode 100644
index 00000000000..8bb3cd02196
@@ -0,0 +1,57 @@
+From cabe916dc694997d4892b58986e73a713d5a2f8d Mon Sep 17 00:00:00 2001
+From: "Miss Islington (bot)"
+ <[email protected]>
+Date: Thu, 16 Aug 2018 15:38:03 -0400
+Subject: [PATCH] [3.6] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787)
+ (#8790)
+
+Read from data socket to avoid "[SSL] shutdown while in init" exception
+during shutdown of the dummy server.
+
+Signed-off-by: Christian Heimes <[email protected]>
+
+
+<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
+https://bugs.python.org/issue34391
+<!-- /issue-number -->
+(cherry picked from commit 1590c393360df059160145e7475754427bfc6680)
+
+
+Co-authored-by: Christian Heimes <[email protected]>
+---
+ Lib/test/test_ftplib.py                                         | 5 +++++
+ Misc/NEWS.d/next/Tests/2018-08-16-18-48-47.bpo-34391.ouNfxC.rst | 1 +
+ 2 files changed, 6 insertions(+)
+ create mode 100644 Misc/NEWS.d/next/Tests/2018-08-16-18-48-47.bpo-34391.ouNfxC.rst
+
+diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py
+index 44dd73aeca..4ff2f71afb 100644
+--- a/Lib/test/test_ftplib.py
++++ b/Lib/test/test_ftplib.py
+@@ -876,18 +876,23 @@ class TestTLS_FTPClass(TestCase):
+         # clear text
+         with self.client.transfercmd('list') as sock:
+             self.assertNotIsInstance(sock, ssl.SSLSocket)
++            self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii'))
+         self.assertEqual(self.client.voidresp(), "226 transfer complete")
+ 
+         # secured, after PROT P
+         self.client.prot_p()
+         with self.client.transfercmd('list') as sock:
+             self.assertIsInstance(sock, ssl.SSLSocket)
++            # consume from SSL socket to finalize handshake and avoid
++            # "SSLError [SSL] shutdown while in init"
++            self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii'))
+         self.assertEqual(self.client.voidresp(), "226 transfer complete")
+ 
+         # PROT C is issued, the connection must be in cleartext again
+         self.client.prot_c()
+         with self.client.transfercmd('list') as sock:
+             self.assertNotIsInstance(sock, ssl.SSLSocket)
++            self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii'))
+         self.assertEqual(self.client.voidresp(), "226 transfer complete")
+ 
+     def test_login(self):
+-- 
+2.11.0
+
meta/recipes-devtools/python/python3_3.5.5.bb +1 -0
index 40896dbc082..93878e4e45c 100644
@@ -42,6 +42,7 @@ SRC_URI += "\
             file://Use-correct-CFLAGS-for-extensions-when-cross-compili.patch \
             file://0002-Makefile-add-target-to-split-profile-generation.patch \
             file://float-endian.patch \
+            file://ftplib.patch \
            "
 SRC_URI[md5sum] = "f3763edf9824d5d3a15f5f646083b6e0"
 SRC_URI[sha256sum] = "063d2c3b0402d6191b90731e0f735c64830e7522348aeb7ed382a83165d45009"
Signature · 1 line
2.11.0
#1 lore raw

PATCH 2/4 python3: remove redundant patch

None of the Python invocations that this changes are actually called, and there's no need to provide a HOSTPGEN variable when the recipe can just override PGEN directly.

---
 .../python/python3-native_3.5.5.bb                 |  1 -
 .../python3/0001-cross-compile-support.patch       | 96 ----------------------
 meta/recipes-devtools/python/python3_3.5.5.bb      |  3 +-
 3 files changed, 1 insertion(+), 99 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
meta/recipes-devtools/python/python3-native_3.5.5.bb +0 -1
index 3098027d803..7b940a16d28 100644
@@ -7,7 +7,6 @@ DISTRO_SRC_URI_linuxstdbase = ""
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 file://12-distutils-prefix-is-inside-staging-area.patch \
 file://python-config.patch \
-file://0001-cross-compile-support.patch \
 file://030-fixup-include-dirs.patch \
 file://070-dont-clean-ipkg-install.patch \
 file://080-distutils-dont_adjust_files.patch \
meta/recipes-devtools/python/python3/0001-cross-compile-support.patch deleted +0 -96

96 lines not on this page · raw message

meta/recipes-devtools/python/python3_3.5.5.bb +1 -2
index 93878e4e45c..6598b486afe 100644
@@ -11,7 +11,6 @@ DISTRO_SRC_URI ?= "file://sitecustomize.py"
 DISTRO_SRC_URI_linuxstdbase = ""
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 file://python-config.patch \
-file://0001-cross-compile-support.patch \
 file://030-fixup-include-dirs.patch \
 file://070-dont-clean-ipkg-install.patch \
 file://080-distutils-dont_adjust_files.patch \
@@ -101,7 +100,7 @@ do_configure_append() {
 }
 
 run_make() {
-	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
+	oe_runmake PGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
 		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
 		STAGING_LIBDIR=${STAGING_LIBDIR} \
 		STAGING_INCDIR=${STAGING_INCDIR} \
Signature · 1 line
2.11.0
#2 lore raw

Re: [PATCH 2/4] python3: remove redundant patch

Quoted 5 lines · Ross Burton

None of the Python invocations that this changes are actually called, and there's no need to provide a HOSTPGEN variable when the recipe can just override PGEN directly.

patch is also substituting ./$(BUILDPYTHON) with $(HOSTPYTHON), how is that taken care of ?
maybe try to a build where hostarch != targetarch might catch the problem. If its handled in other
ways then its probably good riddance.

Quoted 151 lines
---
 .../python/python3-native_3.5.5.bb                 |  1 -
 .../python3/0001-cross-compile-support.patch       | 96 ----------------------
 meta/recipes-devtools/python/python3_3.5.5.bb      |  3 +-
 3 files changed, 1 insertion(+), 99 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/0001-cross-compile-support.patch
meta/recipes-devtools/python/python3-native_3.5.5.bb +0 -1
index 3098027d803..7b940a16d28 100644
@@ -7,7 +7,6 @@ DISTRO_SRC_URI_linuxstdbase = ""
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 file://12-distutils-prefix-is-inside-staging-area.patch \
 file://python-config.patch \
-file://0001-cross-compile-support.patch \
 file://030-fixup-include-dirs.patch \
 file://070-dont-clean-ipkg-install.patch \
 file://080-distutils-dont_adjust_files.patch \
meta/recipes-devtools/python/python3/0001-cross-compile-support.patch deleted +0 -96
deleted file mode 100644
index 7cd7e3b4906..00000000000
@@ -1,96 +0,0 @@
-From ecde3ea170999a9ef734e8af4d7c25be5ba81697 Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <[email protected]>
-Date: Fri, 31 Mar 2017 15:42:46 +0300
-Subject: [PATCH] cross-compile support
-
-We cross compile python. This patch uses tools from host/native
-python instead of in-tree tools
-
--Khem
-
-Rebased on 3.5.4
-
-Upstream-Status: Inappropriate[Configuration Specific]
-Signed-off-by: Alexander Kanavin <[email protected]>
-Signed-off-by: Derek Straka <[email protected]>
----
- Makefile.pre.in | 14 ++++++++------
- 1 file changed, 8 insertions(+), 6 deletions(-)
-
-diff --git a/Makefile.pre.in b/Makefile.pre.in
-index 144c1f8629..f252ac2417 100644
---- a/Makefile.pre.in
-+++ b/Makefile.pre.in
-@@ -223,6 +223,7 @@ LIBOBJS=   @LIBOBJS@
-
- PYTHON=               python$(EXE)
- BUILDPYTHON=  python$(BUILDEXE)
-+HOSTPYTHON=    $(BUILDPYTHON)
-
- PYTHON_FOR_REGEN=@PYTHON_FOR_REGEN@
- PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@
-@@ -277,6 +278,7 @@ LIBFFI_INCLUDEDIR= @LIBFFI_INCLUDEDIR@
- ##########################################################################
- # Parser
- PGEN=         Parser/pgen$(EXE)
-+HOSTPGEN=     $(PGEN)$(EXE)
-
- PSRCS=                \
-               Parser/acceler.c \
-@@ -478,7 +480,7 @@ build_all_generate_profile:
-
- run_profile_task:
-       : # FIXME: can't run for a cross build
--      $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
-+      $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true
-
- build_all_merge_profile:
-       $(LLVM_PROF_MERGER)
-@@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
-       # Regenerate Include/graminit.h and Python/graminit.c
-       # from Grammar/Grammar using pgen
-       @$(MKDIR_P) Include
--      $(PGEN) $(srcdir)/Grammar/Grammar \
-+      $(HOSTPGEN) $(srcdir)/Grammar/Grammar \
-               $(srcdir)/Include/graminit.h \
-               $(srcdir)/Python/graminit.c
-
-@@ -978,7 +980,7 @@ $(LIBRARY_OBJS) $(MODOBJS) Programs/python.o: $(PYTHON_HEADERS)
- ######################################################################
-
- TESTOPTS=     $(EXTRATESTOPTS)
--TESTPYTHON=   $(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
-+TESTPYTHON=   $(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)
- TESTRUNNER=   $(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
- TESTTIMEOUT=  3600
-
-@@ -1470,7 +1472,7 @@ frameworkinstallstructure:       $(LDLIBRARY)
-               fi; \
-       done
-       $(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
--      sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
-+      sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
-       $(LN) -fsn $(VERSION) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/Current
-       $(LN) -fsn Versions/Current/$(PYTHONFRAMEWORK) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/$(PYTHONFRAMEWORK)
-       $(LN) -fsn Versions/Current/Headers $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Headers
-@@ -1543,7 +1545,7 @@ Python/dtoa.o: Python/dtoa.c
-
- # Run reindent on the library
- reindent:
--      ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
-+      $(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
-
- # Rerun configure with the same options as it was run last time,
- # provided the config.status script exists
-@@ -1678,7 +1680,7 @@ funny:
-
- # Perform some verification checks on any modified files.
- patchcheck: @DEF_MAKE_RULE@
--      $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
-+      $(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
-
- # Dependencies
-
---
-2.11.0
-
meta/recipes-devtools/python/python3_3.5.5.bb +1 -3
index 93878e4e45c..6598b486afe 100644
@@ -11,7 +11,6 @@ DISTRO_SRC_URI ?= "file://sitecustomize.py"
 DISTRO_SRC_URI_linuxstdbase = ""
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 file://python-config.patch \
-file://0001-cross-compile-support.patch \
 file://030-fixup-include-dirs.patch \
 file://070-dont-clean-ipkg-install.patch \
 file://080-distutils-dont_adjust_files.patch \
@@ -101,7 +100,7 @@ do_configure_append() {
 }
 
 run_make() {
-       oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
+       oe_runmake PGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
                HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
                STAGING_LIBDIR=${STAGING_LIBDIR} \
                STAGING_INCDIR=${STAGING_INCDIR} \
--

2.11.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#3 lore raw

Re: [PATCH 2/4] python3: remove redundant patch

Quoted 5 lines · Khem Raj

patch is also substituting ./$(BUILDPYTHON) with $(HOSTPYTHON), how is that taken care of ?
maybe try to a build where hostarch != targetarch might catch the problem. If its handled in other
ways then its probably good riddance.

Target binaries have a different loader so they won't run. From what I could tell none of the codepaths this touched were used in the build. I'll review the changes again and either demonstrate why I'm right, or retract my patch.

Ross

#4 lore raw

Re: [PATCH 2/4] python3: remove redundant patch

Line-by-line breakdown of the patch:

Makefile.pre.in LIBOBJS= @LIBOBJS@ · line 225

Quoted 8 lines
Makefile.pre.in +1 -0
index 144c1f8629..f252ac2417 100644
@@ -223,6 +223,7 @@ LIBOBJS= @LIBOBJS@
 PYTHON= python$(EXE)
 BUILDPYTHON= python$(BUILDEXE)
+HOSTPYTHON=    $(BUILDPYTHON)

New variable added.

Makefile.pre.in LIBOBJS= @LIBOBJS@ · line 227

Quoted 2 lines
Makefile.pre.in +1 -0
 PGEN= Parser/pgen$(EXE)
+HOSTPGEN= $(PGEN)$(EXE)

New variable added. My removal simple sets PGEN=/path/to/pgen-native instead.

Makefile.pre.in LIBOBJS= @LIBOBJS@

Quoted 4 lines
Makefile.pre.in +1 -1
 run_profile_task:
  : # FIXME: can't run for a cross build
- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
+ $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true

We never use run_profile_task.

Makefile.pre.in regen-grammar: $(PGEN) · line 777

Quoted 6 lines
Makefile.pre.in +1 -1
@@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
  # Regenerate Include/graminit.h and Python/graminit.c
  # from Grammar/Grammar using pgen
  @$(MKDIR_P) Include
- $(PGEN) $(srcdir)/Grammar/Grammar \
+ $(HOSTPGEN) $(srcdir)/Grammar/Grammar \

As my patch changes PGEN directly, this is redundant.

Makefile.pre.in regen-grammar: $(PGEN) · line 778

Quoted 2 lines
Makefile.pre.in +1 -1
-TESTPYTHON= $(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
+TESTPYTHON= $(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)

TESTPYTHON is only used by the following targets:
- coverage-report
- test
- testall
- testuniversal
- quicktest

We don't invoke any of those during a build.

Makefile.pre.in frameworkinstallstructure: $(LDLIBRARY) · line 1475

Quoted 8 lines
Makefile.pre.in +1 -1
 TESTRUNNER= $(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
 TESTTIMEOUT= 3600
@@ -1470,7 +1472,7 @@ frameworkinstallstructure: $(LDLIBRARY)
  fi; \
  done
  $(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
- sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
+ sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist

This is MacOS-specific.

Makefile.pre.in frameworkinstallstructure: $(LDLIBRARY) · line 1478

Quoted 4 lines
Makefile.pre.in +1 -1
 # Run reindent on the library
 reindent:
- ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
+ $(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib

We never invoke reindent (as this is for maintainers).

Makefile.pre.in funny: · line 1683

Quoted 6 lines
Makefile.pre.in +1 -1
@@ -1678,7 +1680,7 @@ funny:
 # Perform some verification checks on any modified files.
 patchcheck: @DEF_MAKE_RULE@
- $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
+ $(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
 # Dependencies

We never invoke patchcheck (as this is for maintainers).

Ross

#5 lore raw

Re: [PATCH 2/4] python3: remove redundant patch

Thanks for the clarification. It seems to be fine then

Quoted 75 lines · Ross

Line-by-line breakdown of the patch:

Makefile.pre.in LIBOBJS= @LIBOBJS@ · line 225

Quoted 8 lines
Makefile.pre.in +1 -0
index 144c1f8629..f252ac2417 100644
@@ -223,6 +223,7 @@ LIBOBJS= @LIBOBJS@
 PYTHON= python$(EXE)
 BUILDPYTHON= python$(BUILDEXE)
+HOSTPYTHON=    $(BUILDPYTHON)

New variable added.

Makefile.pre.in LIBOBJS= @LIBOBJS@ · line 227

Quoted 2 lines
Makefile.pre.in +1 -0
 PGEN= Parser/pgen$(EXE)
+HOSTPGEN= $(PGEN)$(EXE)

New variable added. My removal simple sets PGEN=/path/to/pgen-native instead.

Makefile.pre.in LIBOBJS= @LIBOBJS@

Quoted 4 lines
Makefile.pre.in +1 -1
 run_profile_task:
  : # FIXME: can't run for a cross build
- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
+ $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true

We never use run_profile_task.

Makefile.pre.in regen-grammar: $(PGEN) · line 777

Quoted 6 lines
Makefile.pre.in +1 -1
@@ -772,7 +774,7 @@ regen-grammar: $(PGEN)
  # Regenerate Include/graminit.h and Python/graminit.c
  # from Grammar/Grammar using pgen
  @$(MKDIR_P) Include
- $(PGEN) $(srcdir)/Grammar/Grammar \
+ $(HOSTPGEN) $(srcdir)/Grammar/Grammar \

As my patch changes PGEN directly, this is redundant.

Makefile.pre.in regen-grammar: $(PGEN) · line 778

Quoted 2 lines
Makefile.pre.in +1 -1
-TESTPYTHON= $(RUNSHARED) ./$(BUILDPYTHON) $(TESTPYTHONOPTS)
+TESTPYTHON= $(RUNSHARED) $(HOSTPYTHON) $(TESTPYTHONOPTS)

TESTPYTHON is only used by the following targets:
- coverage-report
- test
- testall
- testuniversal
- quicktest

We don't invoke any of those during a build.

Makefile.pre.in frameworkinstallstructure: $(LDLIBRARY) · line 1475

Quoted 8 lines
Makefile.pre.in +1 -1
 TESTRUNNER= $(TESTPYTHON) $(srcdir)/Tools/scripts/run_tests.py
 TESTTIMEOUT= 3600
@@ -1470,7 +1472,7 @@ frameworkinstallstructure: $(LDLIBRARY)
  fi; \
  done
  $(LN) -fsn include/python$(LDVERSION) $(DESTDIR)$(prefix)/Headers
- sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
+ sed 's/%VERSION%/'"`$(RUNSHARED) $(HOSTPYTHON) -c 'import platform; print(platform.python_version())'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist

This is MacOS-specific.

Makefile.pre.in frameworkinstallstructure: $(LDLIBRARY) · line 1478

Quoted 4 lines
Makefile.pre.in +1 -1
 # Run reindent on the library
 reindent:
- ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
+ $(HOSTPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib

We never invoke reindent (as this is for maintainers).

Makefile.pre.in funny: · line 1683

Quoted 6 lines
Makefile.pre.in +1 -1
@@ -1678,7 +1680,7 @@ funny:
 # Perform some verification checks on any modified files.
 patchcheck: @DEF_MAKE_RULE@
- $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
+ $(RUNSHARED) $(HOSTPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
 # Dependencies

We never invoke patchcheck (as this is for maintainers).

Ross

#6 lore raw

PATCH 3/4 python3: remove patch that inadvertently disables xattrs

In reply to Ross Burton

This patch dates back to the addition of the Python 3 recipe to oe-core, and as listxattr is never added to supports_follow_symlinks the extended attribute support will never be enabled.

---
 meta/recipes-devtools/python/python3-native_3.5.5.bb    |  1 -
 .../python/python3/shutil-follow-symlink-fix.patch      | 17 -----------------
 meta/recipes-devtools/python/python3_3.5.5.bb           |  1 -
 3 files changed, 19 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/shutil-follow-symlink-fix.patch
meta/recipes-devtools/python/python3-native_3.5.5.bb +0 -1
index 7b940a16d28..b231542f46f 100644
@@ -15,7 +15,6 @@ file://150-fix-setupterm.patch \
 file://python-3.3-multilib.patch \
 file://03-fix-tkinter-detection.patch \
 file://avoid_warning_about_tkinter.patch \
-file://shutil-follow-symlink-fix.patch \
 file://0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch \
 file://sysroot-include-headers.patch \
 file://unixccompiler.patch \
meta/recipes-devtools/python/python3/shutil-follow-symlink-fix.patch deleted +0 -17
deleted file mode 100644
index 802b1c72038..00000000000
@@ -1,17 +0,0 @@
-shutils should consider symlinks
-
--Khem
-
-Upstream-Status: Pending
-
---- a/Lib/shutil.py	2013-01-29 12:31:06.926555779 -0800
-+++ b/Lib/shutil.py	2013-01-29 16:31:39.097554182 -0800
-@@ -132,7 +132,7 @@ def copymode(src, dst, *, follow_symlink
-     st = stat_func(src)
-     chmod_func(dst, stat.S_IMODE(st.st_mode))
- 
--if hasattr(os, 'listxattr'):
-+if hasattr(os, 'listxattr') and os.listxattr in os.supports_follow_symlinks:
-     def _copyxattr(src, dst, *, follow_symlinks=True):
-         """Copy extended filesystem attributes from `src` to `dst`.
- 
meta/recipes-devtools/python/python3_3.5.5.bb +0 -1
index 6598b486afe..02decf470c8 100644
@@ -28,7 +28,6 @@ SRC_URI += "\
             file://cgi_py.patch \
             file://host_include_contamination.patch \
             file://python-3.3-multilib.patch \
-            file://shutil-follow-symlink-fix.patch \
             file://sysroot-include-headers.patch \
             file://unixccompiler.patch \
             file://avoid-ncursesw-include-path.patch \
Signature · 1 line
2.11.0
#7 lore raw

PATCH 4/4 python3: fix multiprocessing

In reply to Ross Burton

glibc 2.28 slightly changed the behaviour of sigaddset() which broke multiprocessing. Backport a patch from Python 3.6 to solve this.

---
 meta/recipes-devtools/python/python3/signal.patch | 56 +++++++++++++++++++++++
 meta/recipes-devtools/python/python3_3.5.5.bb     |  1 +
 2 files changed, 57 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/signal.patch
meta/recipes-devtools/python/python3/signal.patch new file +56 -0
new file mode 100644
index 00000000000..534a097771c
@@ -0,0 +1,56 @@
+Upstream-Status: Backport
+Signed-off-by: Ross Burton <[email protected]>
+
+From 4315389df3c4e8c1f94a18ab11a4b234762132b1 Mon Sep 17 00:00:00 2001
+From: Antoine Pitrou <[email protected]>
+Date: Mon, 23 Apr 2018 22:22:49 +0200
+Subject: [PATCH] [3.6] bpo-33329: Fix multiprocessing regression on newer
+ glibcs (GH-6575) (GH-6582)
+
+Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some
+reserved signal numbers between 1 and NSIG.  The `range(1, NSIG)` idiom
+is commonly used to select all signals for blocking with `pthread_sigmask`.
+So we ignore the sigaddset() return value until we expose sigfillset()
+to provide a better idiom.
+(cherry picked from commit 25038ecfb665bef641abf8cb61afff7505b0e008)
+---
+ .../next/Library/2018-04-23-13-21-39.bpo-33329.lQ-Eod.rst  |  1 +
+ Modules/signalmodule.c                                     | 14 ++++++++------
+ 2 files changed, 9 insertions(+), 6 deletions(-)
+ create mode 100644 Misc/NEWS.d/next/Library/2018-04-23-13-21-39.bpo-33329.lQ-Eod.rst
+
+diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
+index e0d06b434d..138e74e8a9 100644
+--- a/Modules/signalmodule.c
++++ b/Modules/signalmodule.c
+@@ -744,7 +744,6 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
+     int result = -1;
+     PyObject *iterator, *item;
+     long signum;
+-    int err;
+ 
+     sigemptyset(mask);
+ 
+@@ -766,11 +765,14 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
+         Py_DECREF(item);
+         if (signum == -1 && PyErr_Occurred())
+             goto error;
+-        if (0 < signum && signum < NSIG)
+-            err = sigaddset(mask, (int)signum);
+-        else
+-            err = 1;
+-        if (err) {
++        if (0 < signum && signum < NSIG) {
++            /* bpo-33329: ignore sigaddset() return value as it can fail
++             * for some reserved signals, but we want the `range(1, NSIG)`
++             * idiom to allow selecting all valid signals.
++             */
++            (void) sigaddset(mask, (int)signum);
++        }
++        else {
+             PyErr_Format(PyExc_ValueError,
+                          "signal number %ld out of range", signum);
+             goto error;
+-- 
+2.11.0
+
meta/recipes-devtools/python/python3_3.5.5.bb +1 -0
index 02decf470c8..15ec7d7fda6 100644
@@ -41,6 +41,7 @@ SRC_URI += "\
             file://0002-Makefile-add-target-to-split-profile-generation.patch \
             file://float-endian.patch \
             file://ftplib.patch \
+            file://signal.patch \
            "
 SRC_URI[md5sum] = "f3763edf9824d5d3a15f5f646083b6e0"
 SRC_URI[sha256sum] = "063d2c3b0402d6191b90731e0f735c64830e7522348aeb7ed382a83165d45009"
Signature · 1 line
2.11.0
#8 lore raw

✗ patchtest: failure for "python3: fix ftplib with TLS 1..." and 3 more

In reply to Ross Burton

== Series Details ==

Series: "python3: fix ftplib with TLS 1..." and 3 more
Revision: 1
URL : https://patchwork.openembedded.org/series/13994/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at 0191456d81)

* Issue Added patch file is missing Upstream-Status in the header [test_upstream_status_presence_format]

  Suggested fix    Add Upstream-Status: <Valid status> to the header of meta/recipes-devtools/python/python3/ftplib.patch
  Standard format  Upstream-Status: <Valid status>
  Valid status     Pending, Accepted, Backport, Denied, Inappropriate [reason], Submitted [where]

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe