Docker related fixes
cover Docker related fixes
This series updates all the depedencies in requirements.txt, notably a fix for a GitPython vulnerability and fixes enabling latest mysqlclient (v2.2.1) wheel to be buildable.
Also included is a move to the mariadb:lts container in docker-compose, since in practice this is closer to what we are running in production and is the version we should be striving to be compatible with (unless upstream Django has breakage we cannot fix).
The following changes since commit b0af957ac734b76bc7cc2b45b0beb1cde74df0a6:
layerindex: improve updates for actual_branch (2024-01-03 13:33:37 -0800)
are available in the Git repository at:
https://github.com/moto-timo/layerindex-web docker-fixes
https://github.com/moto-timo/layerindex-web/tree/docker-fixes
Tim Orling (6):
dockersetup: use .zst not .zstd
Dockerfile: sort apt packages alphabetically
Dockerfile: fix mysqlclient==2.2.1 build
requirements.txt: bump all to latest
docker-compose.yml: mariadb:lts
docker-compose.yml: drop version '3'
Dockerfile | 31 ++++++++++++++++--------------- docker-compose.yml | 3 +-- dockersetup.py | 4 ++-- requirements.txt | 14 +++++++------- 4 files changed, 26 insertions(+), 26 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/6 dockersetup: use .zst not .zstd
The default file extension for zstd compression is .zst
--- dockersetup.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
dockersetup.py +2 -2
@@ -54,7 +54,7 @@ def get_args(): parser.add_argument('-s', '--https-proxy', type=str, help='https proxy in the format http://<myproxy:port>', default=default_https_proxy, required=False) parser.add_argument('-S', '--socks-proxy', type=str, help='socks proxy in the format socks://myproxy:port>', default=default_socks_proxy, required=False) parser.add_argument('-N', '--no-proxy', type=str, help='Comma-separated list of hosts that should not be connected to via the proxy', default=default_no_proxy, required=False) - parser.add_argument('-d', '--databasefile', type=str, help='Location of your database file to import. Must be a .sql, .sql.gz or .sql.zstd file.', required=False) + parser.add_argument('-d', '--databasefile', type=str, help='Location of your database file to import. Must be a .sql, .sql.gz or .sql.zst file.', required=False) parser.add_argument('-e', '--email-host', type=str, help='Email host for sending messages (optionally with :port if not 25)', required=False) parser.add_argument('--email-user', type=str, help='User name to use when connecting to email host', required=False) parser.add_argument('--email-password', type=str, help='Password to use when connecting to email host', required=False)
@@ -808,7 +808,7 @@ if not args.update: # Import the user's supplied data if args.databasefile: filename, file_extension = os.path.splitext(args.databasefile) - if file_extension == ".zstd": + if file_extension == ".zst": return_code = subprocess.call("zstd -t %s > /dev/null 2>&1" % quote(args.databasefile), shell=True) if return_code == 0: catcmd = 'zstdcat'
Signature · 1 line
2.34.1
PATCH 2/6 Dockerfile: sort apt packages alphabetically
In reply to Tim OrlingMake it is a bit easier to see what is installed by sorting the package list alphabetically.
--- Dockerfile | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
Dockerfile +13 -13
@@ -23,26 +23,26 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \ && update-locale \ && apt-get install -y --no-install-recommends \ autoconf \ + cpio \ + curl \ g++ \ gcc \ - make \ - python2 \ - python3-pip \ - python3-mysqldb \ - python3-dev \ - python3-wheel \ - zlib1g-dev \ + git \ libfreetype6-dev \ libjpeg-dev \ libmariadb-dev-compat \ + make \ netcat-openbsd \ - curl \ - wget \ - git \ - vim \ - rpm2cpio \ + python2 \ + python3-dev \ + python3-mysqldb \ + python3-pip \ + python3-wheel \ rpm \ - cpio \ + rpm2cpio \ + vim \ + wget \ + zlib1g-dev \ && echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \ && locale-gen en_US.UTF-8 \ && update-locale \
Signature · 1 line
2.34.1
PATCH 3/6 Dockerfile: fix mysqlclient==2.2.1 build
In reply to Tim OrlingTo build mysqlclient==2.2.1 we need to add default-libmysqlclient-dev and pkg-config
Drop conflicting libmariadb-dev-compat
Drop python3-mysqldb to allow requirements.txt to override version
Upgrade pip and setuptools before installing from requirements.txt as newer versions are required to properly build mysqlclient wheel.
--- Dockerfile | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
Dockerfile +5 -4
@@ -30,12 +30,12 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \ git \ libfreetype6-dev \ libjpeg-dev \ - libmariadb-dev-compat \ + default-libmysqlclient-dev \ make \ netcat-openbsd \ + pkg-config \ python2 \ python3-dev \ - python3-mysqldb \ python3-pip \ python3-wheel \ rpm \
@@ -46,10 +46,11 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \ && echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \ && locale-gen en_US.UTF-8 \ && update-locale \ + && pip3 install pip --upgrade \ + && pip3 install setuptools --upgrade \ && pip3 install gunicorn \ - && pip3 install setuptools \ && pip3 install -r /requirements.txt \ - && apt-get purge -y autoconf g++ make python3-dev libjpeg-dev \ + && apt-get purge -y autoconf g++ make python3-dev pkg-config libjpeg-dev \ && apt-get autoremove -y \ && rm -rf /var/lib/apt/lists/* \ && apt-get clean
Signature · 1 line
2.34.1
PATCH 4/6 requirements.txt: bump all to latest
In reply to Tim OrlingBump all the dependencies to latest, using
pur -r requirements.txt
We are now able to build the mysqlclient==2.2.1 package, so the only constraint is Django>=4.2,<4.3 to keep us pinned at the desired LTS version.
--- requirements.txt | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
requirements.txt +7 -7
@@ -11,20 +11,20 @@ confusable-homoglyphs==3.2.0 diff-match-patch==20230430 Django>=4.2,<4.3 django-appconf==1.0.6 -django-axes==6.2.0 +django-axes==6.3.0 django-cors-headers==4.3.1 django-ipware==6.0.3 django-ranged-response==0.2.0 django-registration==3.4 -django-reversion==5.0.9 +django-reversion==5.0.10 django-reversion-compare==0.16.2 django-simple-captcha==0.6.0 djangorestframework==3.14.0 gitdb==4.0.11 -GitPython==3.1.40 +GitPython==3.1.41 kombu==5.3.4 -mysqlclient==2.1.1 -Pillow==10.1.0 +mysqlclient==2.2.1 +Pillow==10.2.0 prompt-toolkit==3.0.43 python-dateutil==2.8.2 pytz==2023.3.post1
@@ -33,6 +33,6 @@ smmap==5.0.1 soupsieve==2.5 sqlparse==0.4.4 typing_extensions==4.9.0 -tzdata==2023.3 +tzdata==2023.4 vine==5.1.0 -wcwidth==0.2.12 +wcwidth==0.2.13
Signature · 1 line
2.34.1
PATCH 5/6 docker-compose.yml: mariadb:lts
In reply to Tim OrlingWe should really be striving to be compatible with the latest mariadb LTS (currently 10.11).
--- docker-compose.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
docker-compose.yml +1 -1
@@ -1,7 +1,7 @@ version: '3' services: layersdb: - image: mariadb:10.4 + image: mariadb:lts command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --wait_timeout=28800 --max_allowed_packet=128M environment: - "MYSQL_DATABASE=layersdb"
Signature · 1 line
2.34.1
PATCH 6/6 docker-compose.yml: drop version '3'
In reply to Tim OrlingDrop the obsolete version: '3' property, it is no longer needed since late 2020 (Docker Compose 1.27+) and is only informative:
https://nickjanetakis.com/blog/docker-tip-51-which-docker-compose-api-version-should-you-use
https://github.com/compose-spec/compose-spec/blob/master/04-version-and-name.md
--- docker-compose.yml | 1 - 1 file changed, 1 deletion(-)
docker-compose.yml +0 -1
@@ -1,4 +1,3 @@ -version: '3' services: layersdb: image: mariadb:lts
Signature · 1 line
2.34.1