RFC: Convert to Cargo workspace
Subsystems (3)
cover RFC: Convert to Cargo workspace
The goal of this patch series is to get feedback on converting from a single cargo crate to a cargo workspace. This series is incomplete, and as such, is marked as RFC.
Why using a cargo workspace is better than a single crate:
* Better modularity.
* Since a crate is the smallest unit of compilation in Rust (not individual files), multiple crates avoid recompiling everything
* A C header is created for each crate by cbindgen (if requested), avoiding a single monolithic C header from all Rust sources.
* This separates the dependencies of each crate, which is important if we’re forced to take an old or new dependency for a non-library crate, for example, and don’t want those altered dependencies affecting other crates.
I am particularly interested in feedback on the known issues below:
* github workflow rust-analysis: broken because I removed the rust-version statement in Cargo.toml. The minimum Rust version supported by Git should be documented, but we shouldn’t require building with that minimum version.
* win+Meson build: failing because it can't find the static library
* Rust unit testing: not implemented yet, Make and Meson need to call cargo test
* Conflicts with Brian's "SHA-1/SHA-256 interoperability" patch series
Much of Patrick's earlier work needed to be removed because it assumed a single-crate layout.
The Rust crates are located under rust/, but cargo build must be invoked from the top-level Git directory. cbindgen is included as part of this series, but nothing uses it yet. The generated/ directory is where cbindgen places the generated C header files.
Ezekiel Newren (13):
make: undo Patrick's changes concerning Rust
meson: undo Patrick's changes concerning Rust
cargo: convert from a crate to a workspace
build: build Rust with Makefile and Meson .gitignore: ignore /generated/
cargo: create crate generate-headers
cargo: create crate link-with-c
rust/gitcore: link with c
varint.h: unsigned char -> uint8_t
make: delete files in generated/
github-workflows: unify with rust parameters in make and meson github workflows: install Rust
rust/build-rust.sh: update dir_git_root variable instantiation
.github/workflows/main.yml | 62 ++++++++++++++-
.gitignore | 1 +
Cargo.toml | 17 ++--
Makefile | 118 ++++++++++++++++------------
ci/install-dependencies.sh | 14 ++--
ci/install-rust-toolchain.sh | 30 +++++++
ci/install-rustup.sh | 25 ++++++
ci/make-test-artifacts.sh | 9 +++
ci/run-build-and-tests.sh | 17 +++-
meson.build | 83 ++++++++++++++-----
meson_options.txt | 4 +-
rust/build-crate.sh | 63 +++++++++++++++
rust/cbindgen-template.toml | 13 +++
rust/generate-headers/Cargo.toml | 12 +++
rust/generate-headers/src/main.rs | 44 +++++++++++
rust/gitcore/Cargo.toml | 12 +++
rust/gitcore/build.rs | 7 ++
{src => rust/gitcore/src}/lib.rs | 0
{src => rust/gitcore/src}/varint.rs | 0
rust/link-with-c/Cargo.toml | 9 +++
rust/link-with-c/src/lib.rs | 77 ++++++++++++++++++
shared.mak | 1 -
src/cargo-meson.sh | 39 ---------
src/meson.build | 41 ----------
varint.h | 4 +-
25 files changed, 528 insertions(+), 174 deletions(-)
create mode 100755 ci/install-rust-toolchain.sh
create mode 100755 ci/install-rustup.sh
create mode 100755 rust/build-crate.sh
create mode 100644 rust/cbindgen-template.toml
create mode 100644 rust/generate-headers/Cargo.toml
create mode 100644 rust/generate-headers/src/main.rs
create mode 100644 rust/gitcore/Cargo.toml
create mode 100644 rust/gitcore/build.rs
rename {src => rust/gitcore/src}/lib.rs (100%)
rename {src => rust/gitcore/src}/varint.rs (100%)
create mode 100644 rust/link-with-c/Cargo.toml
create mode 100644 rust/link-with-c/src/lib.rs
delete mode 100755 src/cargo-meson.sh
delete mode 100644 src/meson.buildbase-commit: a99f379adf116d53eb11957af5bab5214915f91d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2110%2Fezekielnewren%2Fcargo-workspace-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2110/ezekielnewren/cargo-workspace-v1
Pull-Request: https://github.com/git/git/pull/2110
Signature · 1 line
gitgitgadget
PATCH 1/13 make: undo Patrick's changes concerning Rust
From: Ezekiel Newren <[email protected]>
--- Makefile | 49 ------------------------------------------------- shared.mak | 1 - 2 files changed, 50 deletions(-)
Makefile +0 -49
@@ -483,14 +483,6 @@ include shared.mak # Define LIBPCREDIR=/foo/bar if your PCRE header and library files are # in /foo/bar/include and /foo/bar/lib directories. # -# == Optional Rust support == -# -# Define WITH_RUST if you want to include features and subsystems written in -# Rust into Git. For now, Rust is still an optional feature of the build -# process. With Git 3.0 though, Rust will always be enabled. -# -# Building Rust code requires Cargo. -# # == SHA-1 and SHA-256 defines == # # === SHA-1 backend ===
@@ -691,7 +683,6 @@ OBJECTS = OTHER_PROGRAMS = PROGRAM_OBJS = PROGRAMS = -RUST_SOURCES = EXCLUDED_PROGRAMS = SCRIPT_PERL = SCRIPT_PYTHON =
@@ -928,18 +919,6 @@ TEST_SHELL_PATH = $(SHELL_PATH) LIB_FILE = libgit.a -ifdef DEBUG -RUST_TARGET_DIR = target/debug -else -RUST_TARGET_DIR = target/release -endif - -ifeq ($(uname_S),Windows) -RUST_LIB = $(RUST_TARGET_DIR)/gitcore.lib -else -RUST_LIB = $(RUST_TARGET_DIR)/libgitcore.a -endif - GITLIBS = common-main.o $(LIB_FILE) EXTLIBS =
@@ -963,15 +942,6 @@ BASIC_LDFLAGS = ARFLAGS = rcs PTHREAD_CFLAGS = -# Rust flags -CARGO_ARGS = -ifndef V -CARGO_ARGS += --quiet -endif -ifndef DEBUG -CARGO_ARGS += --release -endif - # For the 'sparse' target SPARSE_FLAGS ?= -std=gnu99 -D__STDC_NO_VLA__ SP_EXTRA_FLAGS =
@@ -1333,9 +1303,7 @@ LIB_OBJS += urlmatch.o LIB_OBJS += usage.o LIB_OBJS += userdiff.o LIB_OBJS += utf8.o -ifndef WITH_RUST LIB_OBJS += varint.o -endif LIB_OBJS += version.o LIB_OBJS += versioncmp.o LIB_OBJS += walker.o
@@ -1534,9 +1502,6 @@ CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/unit-test.o UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o -RUST_SOURCES += src/lib.rs -RUST_SOURCES += src/varint.rs - GIT-VERSION-FILE: FORCE @OLD=$$(cat $@ 2>/dev/null || :) && \ $(call version_gen,"$(shell pwd)",GIT-VERSION-FILE.in,$@) && \
@@ -1566,14 +1531,6 @@ endif ALL_CFLAGS = $(DEVELOPER_CFLAGS) $(CPPFLAGS) $(CFLAGS) $(CFLAGS_APPEND) ALL_LDFLAGS = $(LDFLAGS) $(LDFLAGS_APPEND) -ifdef WITH_RUST -BASIC_CFLAGS += -DWITH_RUST -GITLIBS += $(RUST_LIB) -ifeq ($(uname_S),Windows) -EXTLIBS += -luserenv -endif -endif - ifdef SANITIZE SANITIZERS := $(foreach flag,$(subst $(comma),$(space),$(SANITIZE)),$(flag)) BASIC_CFLAGS += -fsanitize=$(SANITIZE) -fno-sanitize-recover=$(SANITIZE)
@@ -2963,12 +2920,6 @@ scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) $(LIB_FILE): $(LIB_OBJS) $(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^ -$(RUST_LIB): Cargo.toml $(RUST_SOURCES) - $(QUIET_CARGO)cargo build $(CARGO_ARGS) - -.PHONY: rust -rust: $(RUST_LIB) - export DEFAULT_EDITOR DEFAULT_PAGER Documentation/GIT-EXCLUDED-PROGRAMS: FORCE
shared.mak +0 -1
@@ -56,7 +56,6 @@ ifndef V QUIET_MKDIR_P_PARENT = @echo ' ' MKDIR -p $(@D); ## Used in "Makefile" - QUIET_CARGO = @echo ' ' CARGO $@; QUIET_CC = @echo ' ' CC $@; QUIET_AR = @echo ' ' AR $@; QUIET_LINK = @echo ' ' LINK $@;
Signature · 1 line
gitgitgadget
PATCH 2/13 meson: undo Patrick's changes concerning Rust
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- meson.build | 19 ++----------------- meson_options.txt | 4 ++-- src/cargo-meson.sh | 39 --------------------------------------- src/meson.build | 41 ----------------------------------------- 4 files changed, 4 insertions(+), 99 deletions(-) delete mode 100755 src/cargo-meson.sh delete mode 100644 src/meson.build
meson.build +2 -17
@@ -220,7 +220,7 @@ project('git', 'c', # learned to define __STDC_VERSION__ with C11 and later. We thus require # GNU C99 and fall back to C11. Meson only learned to handle the fallback # with version 1.3.0, so on older versions we use GNU C99 unconditionally. - default_options: meson.version().version_compare('>=1.3.0') ? ['rust_std=2018', 'c_std=gnu99,c11'] : ['rust_std=2018', 'c_std=gnu99'], + default_options: meson.version().version_compare('>=1.3.0') ? ['c_std=gnu99,c11'] : ['c_std=gnu99'], ) fs = import('fs')
@@ -1709,21 +1709,6 @@ version_def_h = custom_target( ) libgit_sources += version_def_h -cargo = find_program('cargo', dirs: program_path, native: true, required: get_option('rust')) -rust_option = get_option('rust').disable_auto_if(not cargo.found()) -if rust_option.allowed() - subdir('src') - libgit_c_args += '-DWITH_RUST' - - if host_machine.system() == 'windows' - libgit_dependencies += compiler.find_library('userenv') - endif -else - libgit_sources += [ - 'varint.c', - ] -endif - libgit = declare_dependency( link_with: static_library('git', sources: libgit_sources,
@@ -2270,7 +2255,7 @@ summary({ 'pcre2': pcre2, 'perl': perl_features_enabled, 'python': target_python.found(), - 'rust': rust_option.allowed(), + 'rust': get_option('with_rust'), }, section: 'Auto-detected features', bool_yn: true) summary({
meson_options.txt +2 -2
@@ -71,8 +71,8 @@ option('zlib_backend', type: 'combo', choices: ['auto', 'zlib', 'zlib-ng'], valu # Build tweaks. option('breaking_changes', type: 'boolean', value: false, description: 'Enable upcoming breaking changes.') -option('rust', type: 'feature', value: 'auto', - description: 'Enable building with Rust.') +option('with_rust', type: 'boolean', value: true, + description: 'Enable building with Rust, true by default.') option('macos_use_homebrew_gettext', type: 'boolean', value: true, description: 'Use gettext from Homebrew instead of the slightly-broken system-provided one.')
src/cargo-meson.sh deleted +0 -39
@@ -1,39 +0,0 @@ -#!/bin/sh - -if test "$#" -lt 2 -then - exit 1 -fi - -SOURCE_DIR="$1" -BUILD_DIR="$2" -BUILD_TYPE=debug - -shift 2 - -for arg -do - case "$arg" in - --release) - BUILD_TYPE=release;; - esac -done - -cargo build --lib --quiet --manifest-path="$SOURCE_DIR/Cargo.toml" --target-dir="$BUILD_DIR" "$@" -RET=$? -if test $RET -ne 0 -then - exit $RET -fi - -case "$(cargo -vV | sed -s 's/^host: \(.*\)$/\1/')" in - *-windows-*) - LIBNAME=gitcore.lib;; - *) - LIBNAME=libgitcore.a;; -esac - -if ! cmp "$BUILD_DIR/$BUILD_TYPE/$LIBNAME" "$BUILD_DIR/libgitcore.a" >/dev/null 2>&1 -then - cp "$BUILD_DIR/$BUILD_TYPE/$LIBNAME" "$BUILD_DIR/libgitcore.a" -fi
src/meson.build deleted +0 -41
41 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH 3/13 cargo: convert from a crate to a workspace
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
---
Cargo.toml | 15 +++++----------
rust/gitcore/Cargo.toml | 10 ++++++++++
{src => rust/gitcore/src}/lib.rs | 0
{src => rust/gitcore/src}/varint.rs | 0
4 files changed, 15 insertions(+), 10 deletions(-)
create mode 100644 rust/gitcore/Cargo.toml
rename {src => rust/gitcore/src}/lib.rs (100%)
rename {src => rust/gitcore/src}/varint.rs (100%)Cargo.toml +5 -10
@@ -1,10 +1,5 @@ -[package] -name = "gitcore" -version = "0.1.0" -edition = "2018" -rust-version = "1.49.0" - -[lib] -crate-type = ["staticlib"] - -[dependencies] +[workspace] +members = [ + "rust/gitcore", +] +resolver = "2"
rust/gitcore/Cargo.toml new file +10 -0
@@ -0,0 +1,10 @@ +[package] +name = "gitcore" +version = "0.1.0" +edition = "2018" +rust-version = "1.49.0" + +[lib] +crate-type = ["staticlib"] + +[dependencies]
rust/gitcore/src/lib.rs renamed from src/lib.rs +0 -0
rust/gitcore/src/varint.rs renamed from src/varint.rs +0 -0
Signature · 1 line
gitgitgadget
PATCH 4/13 build: build Rust with Makefile and Meson
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- Makefile | 91 ++++++++++++++++++++++++++++++++++++++------- meson.build | 63 ++++++++++++++++++++++++++++++- rust/build-crate.sh | 58 +++++++++++++++++++++++++++++ 3 files changed, 197 insertions(+), 15 deletions(-) create mode 100755 rust/build-crate.sh
Makefile +78 -13
@@ -483,6 +483,14 @@ include shared.mak # Define LIBPCREDIR=/foo/bar if your PCRE header and library files are # in /foo/bar/include and /foo/bar/lib directories. # +# == Optional Rust support == +# +# Define WITH_RUST=false if you don't want to include features and subsystems +# written in Rust into Git. For now, Rust is still an optional feature of the +# build process. With Git 3.0 though, Rust will be mandatory. +# +# Building Rust code requires Cargo. +# # == SHA-1 and SHA-256 defines == # # === SHA-1 backend ===
@@ -919,9 +927,55 @@ TEST_SHELL_PATH = $(SHELL_PATH) LIB_FILE = libgit.a -GITLIBS = common-main.o $(LIB_FILE) EXTLIBS = +GIT_BUILD_DIR := $(CURDIR) +export GIT_BUILD_DIR + +RUST_CRATES := gitcore +.PHONY: rust-compile rust-clean + +WITH_RUST ?= true +ifeq ($(WITH_RUST),true) + +ifeq ($(DEBUG), 1) + RUST_BUILD_MODE := debug +else + RUST_BUILD_MODE := release +endif + +RUST_LIBS := $(foreach c,$(RUST_CRATES),$(GIT_BUILD_DIR)/lib$(c).a) + +rust-compile: + @for c in $(RUST_CRATES); do \ + echo "Building $$c..."; \ + ./rust/build-crate.sh $(GIT_BUILD_DIR) $(RUST_BUILD_MODE) $$c || exit $$?; \ + done + +rust-clean: + $(RM) $(RUST_LIBS) Cargo.lock + cargo clean + +$(GIT_BUILD_DIR)/lib%.a: + echo $(RUST_LIBS) + ./rust/build-crate.sh $(GIT_BUILD_DIR) $(RUST_BUILD_MODE) $* + +UNAME_S := $(shell uname -s) +ifeq ($(UNAME_S),Linux) + EXTLIBS += -ldl +endif + +else ifeq ($(WITH_RUST),false) +rust-compile: + : +rust-clean: + : +else +$(error 'WITH_RUST' must be true or false) +endif + +GITLIBS = common-main.o $(LIB_FILE) + GIT_USER_AGENT = git/$(GIT_VERSION) ifeq ($(wildcard sha1collisiondetection/lib/sha1.h),sha1collisiondetection/lib/sha1.h)
@@ -938,6 +992,11 @@ CC_LD_DYNPATH = -Wl,-rpath, BASIC_CFLAGS = -I. BASIC_LDFLAGS = +ifeq ($(WITH_RUST),true) + BASIC_CFLAGS += -DWITH_RUST + BASIC_CFLAGS += -fPIE +endif + # library flags ARFLAGS = rcs PTHREAD_CFLAGS =
@@ -1303,7 +1362,9 @@ LIB_OBJS += urlmatch.o LIB_OBJS += usage.o LIB_OBJS += userdiff.o LIB_OBJS += utf8.o +ifeq ($(WITH_RUST),false) LIB_OBJS += varint.o +endif LIB_OBJS += version.o LIB_OBJS += versioncmp.o LIB_OBJS += walker.o
@@ -2409,6 +2470,10 @@ endif # from the dependency list, that would make each entry appear twice. LIBS = $(filter-out %.o, $(GITLIBS)) $(EXTLIBS) +ifeq ($(WITH_RUST),true) +LIBS += $(RUST_LIBS) +endif + BASIC_CFLAGS += $(COMPAT_CFLAGS) LIB_OBJS += $(COMPAT_OBJS)
@@ -2564,7 +2629,7 @@ git.sp git.s git.o: EXTRA_CPPFLAGS = \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \ '-DGIT_INFO_PATH="$(infodir_relative_SQ)"' -git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) +git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \ $(filter %.o,$^) $(LIBS)
@@ -2889,17 +2954,17 @@ headless-git.o: compat/win32/headless.c GIT-CFLAGS headless-git$X: headless-git.o git.res GIT-LDFLAGS $(QUIET_LINK)$(CC) $(ALL_CFLAGS) $(ALL_LDFLAGS) -mwindows -o $@ $< git.res -git-%$X: %.o GIT-LDFLAGS $(GITLIBS) +git-%$X: %.o GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS) -git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) +git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(IMAP_SEND_LDFLAGS) $(LIBS) -git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS) +git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(CURL_LIBCURL) $(LIBS) -git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS) +git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)
@@ -2909,11 +2974,11 @@ $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY) ln -s $< $@ 2>/dev/null || \ cp $< $@ -$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS) +$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS) -scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) +scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \ $(filter %.o,$^) $(LIBS)
@@ -3286,7 +3351,7 @@ perf: all t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o -t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) +t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) rust-compile $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS) check-sha1:: t/helper/test-tool$X
@@ -3751,7 +3816,7 @@ cocciclean: $(RM) -r .build/contrib/coccinelle $(RM) contrib/coccinelle/*.cocci.patch -clean: profile-clean coverage-clean cocciclean +clean: profile-clean coverage-clean cocciclean rust-clean $(RM) -r .build $(UNIT_TEST_BIN) $(RM) GIT-TEST-SUITES $(RM) po/git.pot po/git-core.pot
@@ -3907,13 +3972,13 @@ FUZZ_CXXFLAGS ?= $(ALL_CFLAGS) .PHONY: fuzz-all fuzz-all: $(FUZZ_PROGRAMS) -$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS +$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS rust-compile $(QUIET_LINK)$(FUZZ_CXX) $(FUZZ_CXXFLAGS) -o $@ $(ALL_LDFLAGS) \ -Wl,--allow-multiple-definition \ $(filter %.o,$^) $(filter %.a,$^) $(LIBS) $(LIB_FUZZING_ENGINE) $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_OBJS) \ - $(GITLIBS) GIT-LDFLAGS + $(GITLIBS) GIT-LDFLAGS rust-compile $(call mkdir_p_parent_template) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \ $(filter %.o,$^) $(filter %.a,$^) $(LIBS)
@@ -3932,7 +3997,7 @@ $(UNIT_TEST_DIR)/clar.suite: $(UNIT_TEST_DIR)/clar-decls.h $(UNIT_TEST_DIR)/gene $(UNIT_TEST_DIR)/clar/clar.o: $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS): $(UNIT_TEST_DIR)/clar-decls.h $(CLAR_TEST_OBJS): EXTRA_CPPFLAGS = -I$(UNIT_TEST_DIR) -$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS +$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS rust-compile $(call mkdir_p_parent_template) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
meson.build +61 -2
99 lines not on this page · raw message
rust/build-crate.sh new file +58 -0
58 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH 5/13 .gitignore: ignore /generated/
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- .gitignore | 1 + Makefile | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
.gitignore +1 -0
@@ -258,3 +258,4 @@ Release/ /contrib/buildsystems/out /contrib/libgit-rs/target /contrib/libgit-sys/target +/generated/
Makefile +1 -1
@@ -953,7 +953,7 @@ rust-compile: done rust-clean: - $(RM) $(RUST_LIBS) Cargo.lock + $(RM) $(RUST_LIBS) Cargo.lock generated/ cargo clean $(GIT_BUILD_DIR)/lib%.a:
Signature · 1 line
gitgitgadget
PATCH 6/13 cargo: create crate generate-headers
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- Cargo.toml | 1 + rust/cbindgen-template.toml | 13 +++++++++ rust/generate-headers/Cargo.toml | 12 +++++++++ rust/generate-headers/src/main.rs | 44 +++++++++++++++++++++++++++++++ 4 files changed, 70 insertions(+) create mode 100644 rust/cbindgen-template.toml create mode 100644 rust/generate-headers/Cargo.toml create mode 100644 rust/generate-headers/src/main.rs
Cargo.toml +1 -0
@@ -1,5 +1,6 @@ [workspace] members = [ + "rust/generate-headers", "rust/gitcore", ] resolver = "2"
rust/cbindgen-template.toml new file +13 -0
@@ -0,0 +1,13 @@ +## don't include any system files, not even git-compat-util.h +# sys_includes = [] + +autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */" + +language = "C" +no_includes = true +usize_is_size_t = true +style = "tag" +tab_width = 8 + +[parse] +parse_deps = false
rust/generate-headers/Cargo.toml new file +12 -0
@@ -0,0 +1,12 @@ +[package] +name = "generate-headers" +version = "0.1.0" +edition = "2018" + +[dependencies] +cbindgen = "0.24.0" +textwrap = "=0.16.1" +once_cell = "=1.20.3" +unicode-width = "=0.1.13" +quote = "=1.0.41" +syn = "=2.0.106"
rust/generate-headers/src/main.rs new file +44 -0
44 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH 7/13 cargo: create crate link-with-c
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- rust/link-with-c/Cargo.toml | 9 +++++ rust/link-with-c/src/lib.rs | 77 +++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 rust/link-with-c/Cargo.toml create mode 100644 rust/link-with-c/src/lib.rs
rust/link-with-c/Cargo.toml new file +9 -0
@@ -0,0 +1,9 @@ +[package] +name = "link-with-c" +version = "0.1.0" +edition = "2018" + +[lib] +crate-type = ["staticlib", "rlib"] + +[dependencies]
rust/link-with-c/src/lib.rs new file +77 -0
77 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH 8/13 rust/gitcore: link with c
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- Cargo.toml | 1 + rust/gitcore/Cargo.toml | 6 ++++-- rust/gitcore/build.rs | 7 +++++++ 3 files changed, 12 insertions(+), 2 deletions(-) create mode 100644 rust/gitcore/build.rs
Cargo.toml +1 -0
@@ -2,5 +2,6 @@ members = [ "rust/generate-headers", "rust/gitcore", + "rust/link-with-c", ] resolver = "2"
rust/gitcore/Cargo.toml +4 -2
@@ -2,9 +2,11 @@ name = "gitcore" version = "0.1.0" edition = "2018" -rust-version = "1.49.0" [lib] -crate-type = ["staticlib"] +crate-type = ["staticlib", "rlib"] [dependencies] + +[build-dependencies] +link-with-c = { path = "../link-with-c" }
rust/gitcore/build.rs new file +7 -0
@@ -0,0 +1,7 @@ +use link_with_c::BuildHelper ; + +fn main () { +BuildHelper ::new (std::env::vars ().collect ()) +// .generate_header(|_|{}) + .build (); +}
Signature · 1 line
gitgitgadget
PATCH 9/13 varint.h: unsigned char -> uint8_t
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- varint.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
varint.h +2 -2
@@ -1,7 +1,7 @@#ifndef VARINT_H #define VARINT_H -uint8_t encode_varint (uint64_t ,unsigned char *); -uint64_t decode_varint (const unsigned char **); +uint8_t encode_varint (uint64_t ,uint8_t *); +uint64_t decode_varint (const uint8_t **);#endif /* VARINT_H */
Signature · 1 line
gitgitgadget
PATCH 10/13 make: delete files in generated/
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
@@ -953,7 +953,7 @@ rust-compile: done rust-clean: - $(RM) $(RUST_LIBS) Cargo.lock generated/ + $(RM) $(RUST_LIBS) Cargo.lock generated/*.h cargo clean $(GIT_BUILD_DIR)/lib%.a:
Signature · 1 line
gitgitgadget
PATCH 11/13 github-workflows: unify with rust parameters in make and meson
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
-Drust=enabled -> -Dwith_rust=enabled
WITH_RUST=YesPlease -> WITH_RUST=true
--- ci/run-build-and-tests.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
ci/run-build-and-tests.sh +2 -2
@@ -8,9 +8,9 @@ case "$jobname" in fedora-breaking-changes-musl|linux-breaking-changes) export WITH_BREAKING_CHANGES=YesPlease - export WITH_RUST=YesPlease + export WITH_RUST=true MESONFLAGS="$MESONFLAGS -Dbreaking_changes=true" - MESONFLAGS="$MESONFLAGS -Drust=enabled" + MESONFLAGS="$MESONFLAGS -Dwith_rust=enabled" ;; linux-TEST-vars) export OPENSSL_SHA1_UNSAFE=YesPlease
Signature · 1 line
gitgitgadget
PATCH 12/13 github workflows: install Rust
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- .github/workflows/main.yml | 62 +++++++++++++++++++++++++++++++++++- ci/install-dependencies.sh | 14 ++++---- ci/install-rust-toolchain.sh | 30 +++++++++++++++++ ci/install-rustup.sh | 25 +++++++++++++++ ci/make-test-artifacts.sh | 9 ++++++ ci/run-build-and-tests.sh | 13 ++++++++ 6 files changed, 145 insertions(+), 8 deletions(-) create mode 100755 ci/install-rust-toolchain.sh create mode 100755 ci/install-rustup.sh
.github/workflows/main.yml +61 -1
@@ -26,6 +26,13 @@ jobs: outputs: enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }} skip_concurrent: ${{ steps.check-ref.outputs.skip_concurrent }} + rust_version_minimum: 1.63.0 + rust_version_windows: 1.78.0 + rust_version_musl: 1.72.0 + ## the rust target is inferred by rustup unless specified + rust_target_windows_make: x86_64-pc-windows-gnu + rust_target_windows_meson: x86_64-pc-windows-msvc + rust_target_32bit_linux: i686-unknown-linux-gnu steps: - name: try to clone ci-config branch run: |
@@ -108,12 +115,28 @@ jobs: needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' runs-on: windows-latest + env: + CARGO_HOME: "/c/Users/runneradmin/.cargo" concurrency: group: windows-build-${{ github.ref }} cancel-in-progress: ${{ needs.ci-config.outputs.skip_concurrent == 'yes' }} steps: - uses: actions/checkout@v5 - uses: git-for-windows/setup-git-for-windows-sdk@v1 + with: + flavor: full + - name: Install rustup via github actions + uses: actions-rs/toolchain@v1 + with: + toolchain: stable + profile: minimal + override: false + - name: Install Rust toolchain + shell: bash + env: + RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_windows }} + RUST_TARGET: ${{ needs.ci-config.outputs.rust_target_windows_make }} + run: ci/install-rust-toolchain.sh - name: build shell: bash env:
@@ -254,12 +277,26 @@ jobs: needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' runs-on: windows-latest + env: + CARGO_HOME: "/c/Users/runneradmin/.cargo" concurrency: group: windows-meson-build-${{ github.ref }} cancel-in-progress: ${{ needs.ci-config.outputs.skip_concurrent == 'yes' }} steps: - uses: actions/checkout@v5 - uses: actions/setup-python@v6 + - name: Install rustup via github actions + uses: actions-rs/toolchain@v1 + with: + toolchain: stable + profile: minimal + override: false + - name: Install Rust toolchain + shell: bash + env: + RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_windows }} + RUST_TARGET: ${{ needs.ci-config.outputs.rust_target_windows_meson }} + run: ci/install-rust-toolchain.sh - name: Set up dependencies shell: pwsh run: pip install meson ninja
@@ -329,11 +366,24 @@ jobs: jobname: ${{matrix.vector.jobname}} CI_JOB_IMAGE: ${{matrix.vector.pool}} TEST_OUTPUT_DIRECTORY: ${{github.workspace}}/t + CARGO_HOME: "/Users/runner/.cargo" runs-on: ${{matrix.vector.pool}} steps: - uses: actions/checkout@v5 - run: ci/install-dependencies.sh - - run: ci/run-build-and-tests.sh + - name: Install rustup via github actions + uses: actions-rs/toolchain@v1 + with: + toolchain: stable + profile: minimal + override: false + - name: Install Rust toolchain + shell: bash + env: + RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_minimum }} + run: ci/install-rust-toolchain.sh + - name: Run build and tests + run: ci/run-build-and-tests.sh - name: print test failures if: failure() && env.FAILED_TEST_ARTIFACTS != '' run: ci/print-test-failures.sh
@@ -395,9 +445,11 @@ jobs: cc: gcc - jobname: linux-musl-meson image: alpine:latest + rust_version_override: ${{ needs.ci-config.outputs.rust_version_musl }} # Supported until 2025-04-02. - jobname: linux32 image: i386/ubuntu:focal + rust_target_override: ${{ needs.ci-config.outputs.rust_target_32bit_linux }} # A RHEL 8 compatible distro. Supported until 2029-05-31. - jobname: almalinux-8 image: almalinux:8
@@ -409,6 +461,7 @@ jobs: CC: ${{matrix.vector.cc}} CI_JOB_IMAGE: ${{matrix.vector.image}} CUSTOM_PATH: /custom + CARGO_HOME: /home/builder/.cargo runs-on: ubuntu-latest container: ${{matrix.vector.image}} steps:
@@ -433,6 +486,13 @@ jobs: - run: ci/install-dependencies.sh - run: useradd builder --create-home - run: chown -R builder . + - name: Install rustup via script + run: sudo --preserve-env --set-home --user=builder ci/install-rustup.sh + - name: Install Rust toolchain + env: + RUST_VERSION: ${{ matrix.vector.rust_version_override || needs.ci-config.outputs.rust_version_minimum }} + RUST_TARGET: ${{ matrix.vector.rust_target_override || '' }} + run: sudo --preserve-env --set-home --user=builder ci/install-rust-toolchain.sh - run: sudo --preserve-env --set-home --user=builder ci/run-build-and-tests.sh - name: print test failures if: failure() && env.FAILED_TEST_ARTIFACTS != ''
ci/install-dependencies.sh +7 -7
@@ -26,7 +26,7 @@ fi case "$distro" in alpine-*) - apk add --update shadow sudo meson ninja-build gcc libc-dev curl-dev openssl-dev expat-dev gettext \ + apk add --update shadow sudo meson ninja-build gcc libc-dev curl curl-dev openssl-dev expat-dev gettext \ zlib-ng-dev pcre2-dev python3 musl-libintl perl-utils ncurses \ apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \ bash cvs gnupg perl-cgi perl-dbd-sqlite perl-io-tty >/dev/null
@@ -37,7 +37,7 @@ fedora-*|almalinux-*) MESON_DEPS="meson ninja";; esac dnf -yq update >/dev/null && - dnf -yq install shadow-utils sudo make pkg-config gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel $MESON_DEPS cargo >/dev/null + dnf -yq install shadow-utils sudo make pkg-config gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl curl-devel pcre2-devel $MESON_DEPS cargo >/dev/null ;; ubuntu-*|i386/ubuntu-*|debian-*) # Required so that apt doesn't wait for user input on certain packages.
@@ -61,8 +61,8 @@ ubuntu-*|i386/ubuntu-*|debian-*) sudo apt-get -q update sudo apt-get -q -y install \ $LANGUAGES apache2 cvs cvsps git gnupg $SVN \ - make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \ - tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ + make libssl-dev curl libcurl4-openssl-dev libexpat-dev wget sudo default-jre \ + tcl tk gettext zlib1g zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ libemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ libsecret-1-dev libpcre2-dev meson ninja-build pkg-config cargo \ ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE
@@ -134,7 +134,7 @@ ClangFormat) sudo apt-get -q -y install clang-format ;; StaticAnalysis) - sudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \ + sudo apt-get -q -y install coccinelle curl libcurl4-openssl-dev libssl-dev \ libexpat-dev gettext make ;; RustAnalysis)
@@ -149,8 +149,8 @@ RustAnalysis) sudo chmod a+x "$CUSTOM_PATH/cargo-msrv" ;; sparse) - sudo apt-get -q -y install libssl-dev libcurl4-openssl-dev \ - libexpat-dev gettext zlib1g-dev sparse + sudo apt-get -q -y install libssl-dev curl libcurl4-openssl-dev \ + libexpat-dev gettext zlib1g zlib1g-dev sparse ;; Documentation) sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
ci/install-rust-toolchain.sh new file +30 -0
@@ -0,0 +1,30 @@ +#!/bin/sh + +if [ "$CARGO_HOME" = "" ]; then + echo >&2 "::error:: CARGO_HOME is not set" + exit 2 +fi +export PATH="$CARGO_HOME/bin:$PATH" +rustup -vV || exit $? + +## Enforce the correct Rust toolchain +rustup override unset || true + +## install a specific version of rust +if [ "$RUST_TARGET" != "" ]; then + rustup default --force-non-host "$RUST_VERSION-$RUST_TARGET" || exit $? +else + rustup default "$RUST_VERSION" || exit $? +fi + +rustc -vV || exit $? + +RE_RUST_TARGET="$RUST_TARGET" +if [ "$RUST_TARGET" = "" ]; then + RE_RUST_TARGET="[^ ]+" +fi + +if ! rustup show active-toolchain | grep -E "^$RUST_VERSION-$RE_RUST_TARGET \(default\)$"; then + echo >&2 "::error:: wrong Rust toolchain, active-toolchain: $(rustup show active-toolchain)" + exit 3 +fi
ci/install-rustup.sh new file +25 -0
@@ -0,0 +1,25 @@ +#!/bin/sh + +## github workflows actions-rs/toolchain@v1 doesn't work for docker +## targets. This script should only be used if the ci pipeline +## doesn't support installing rust on a particular target. + +if [ "$(id -u)" -eq 0 ]; then + echo >&2 "::warning:: installing rust as root" +fi + +if [ "$CARGO_HOME" = "" ]; then + echo >&2 "::error:: CARGO_HOME is not set" + exit 2 +fi + +export RUSTUP_HOME=$CARGO_HOME + +## install rustup +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y +if [ ! -f $CARGO_HOME/env ]; then + echo "PATH=$CARGO_HOME/bin:\$PATH" > $CARGO_HOME/env +fi +. $CARGO_HOME/env + +rustup -vV
ci/make-test-artifacts.sh +9 -0
@@ -7,6 +7,15 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early . ${0%/*}/lib.sh +## ensure rustup is in the PATH variable +if [ "$CARGO_HOME" = "" ]; then + echo >&2 "::error:: CARGO_HOME is not set" + exit 2 +fi +export PATH="$CARGO_HOME/bin:$PATH" + +rustc -vV + group Build make artifacts-tar ARTIFACTS_DIRECTORY="$1" check_unignored_build_artifacts
ci/run-build-and-tests.sh +13 -0
@@ -5,6 +5,15 @@ . ${0%/*}/lib.sh +## ensure rustup is in the PATH variable +if [ "$CARGO_HOME" = "" ]; then + echo >&2 "::error:: CARGO_HOME is not set" + exit 2 +fi +. $CARGO_HOME/env + +rustc -vV || exit $? + case "$jobname" in fedora-breaking-changes-musl|linux-breaking-changes) export WITH_BREAKING_CHANGES=YesPlease
@@ -60,5 +69,9 @@ case "$jobname" in ;; esac +if [ -d "$CARGO_HOME" ]; then + rm -rf $CARGO_HOME +fi + check_unignored_build_artifacts save_good_tree
Signature · 1 line
gitgitgadget
PATCH 13/13 rust/build-rust.sh: update dir_git_root variable instantiation
In reply to Ezekiel Newren via GitGitGadgetFrom: Ezekiel Newren <[email protected]>
--- Makefile | 6 ++++-- meson.build | 1 + rust/build-crate.sh | 15 ++++++++++----- 3 files changed, 15 insertions(+), 7 deletions(-)
Makefile +4 -2
@@ -931,6 +931,8 @@ EXTLIBS = GIT_BUILD_DIR := $(CURDIR) export GIT_BUILD_DIR +MAKEFILE_DIR := $(dir $(lastword $(MAKEFILE_LIST))) +export MAKEFILE_DIR RUST_CRATES := gitcore .PHONY: rust-compile rust-clean
@@ -949,7 +951,7 @@ RUST_LIBS := $(foreach c,$(RUST_CRATES),$(GIT_BUILD_DIR)/lib$(c).a) rust-compile: @for c in $(RUST_CRATES); do \ echo "Building $$c..."; \ - ./rust/build-crate.sh $(GIT_BUILD_DIR) $(RUST_BUILD_MODE) $$c || exit $$?; \ + ./rust/build-crate.sh $(MAKEFILE_DIR) $(GIT_BUILD_DIR) $(RUST_BUILD_MODE) $$c || exit $$?; \ done rust-clean:
@@ -958,7 +960,7 @@ rust-clean: $(GIT_BUILD_DIR)/lib%.a: echo $(RUST_LIBS) - ./rust/build-crate.sh $(GIT_BUILD_DIR) $(RUST_BUILD_MODE) $* + ./rust/build-crate.sh $(MAKEFILE_DIR) $(GIT_BUILD_DIR) $(RUST_BUILD_MODE) $* UNAME_S := $(shell uname -s) ifeq ($(UNAME_S),Linux)
meson.build +1 -0
@@ -305,6 +305,7 @@ if with_rust build_always_stale: true, command: [ meson.project_source_root() / 'rust' / 'build-crate.sh', + meson.project_source_root(), meson.current_build_dir(), rust_build_profile, crate, ], install: false,
rust/build-crate.sh +10 -5
@@ -3,10 +3,12 @@ rustc -vV || exit $? cargo --version || exit $? -dir_git_root=${0%/*} -dir_build=$1 -rust_build_profile=$2 -crate=$3 +dir_git_root=$1 +cd $dir_git_root || exit $? +dir_git_root=$PWD +dir_build=$2 +rust_build_profile=$3 +crate=$4 if [ "$dir_git_root" = "" ]; then echo "did not specify the directory for the root of git"
@@ -38,9 +40,12 @@ libfile="lib${crate}.a" if rustc -vV | grep windows-msvc; then libfile="${crate}.lib" PATH="$(echo $PATH | tr ':' '\n' | grep -Ev "^(/mingw64/bin|/usr/bin)$" | paste -sd: -):/mingw64/bin:/usr/bin" + echo "PATH=$PATH" fi -CARGO_TARGET_DIR=$dir_git_root/.build/rust/$crate +echo "libfile=$libfile" + +CARGO_TARGET_DIR=$dir_build/.build/rust/$crate export CARGO_TARGET_DIR cargo clean && pwd && USE_LINKING="false" cargo build -p $crate $rust_args
Signature · 1 line
gitgitgadget