package/llama-cpp: new package
PATCH 1/1 package/llama-cpp: new package
Add a package for llama.cpp, a C/C++ LLM inference library, used in popular projects like Ollama, RamaLama, and more.
---
This patch adds a package for llama.cpp, an LLM inference library, supporting many popular models, including LLaMa, Gemma, Deepseek, Qwen, and many more.
This library includes tools that can be used for standalone inference, like llama-cli and llama-server, as well as benchmarking, in llama-bench. The library has a variety of software and hardware accelerated backends, but this patch focuses on OpenBLAS and Vulkan support to start.
The patch disables building for S390X and RISCV due to compilation errors related to intrinsics, even though they're technically supported. Older GCC versions are also problematic, and GCC > v9 is required.
As mentioned in the virglrenderer series, this package was used to test virglrenderer with venus in a nested qemu guest using virtio-gpu.
https://lists.buildroot.org/pipermail/buildroot/2025-June/781395.html
The package was tested against all toolchains with utils/test-pkg:
40 builds, 28 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
Feedback welcome.
Thanks,
Joseph
--- DEVELOPERS | 1 + package/Config.in | 1 + package/llama-cpp/Config.in | 36 +++++++++++++++++++++++++++++++++ package/llama-cpp/llama-cpp.hash | 4 ++++ package/llama-cpp/llama-cpp.mk | 43 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 85 insertions(+)
DEVELOPERS +1 -0
@@ -1762,6 +1762,7 @@ F: package/at-spi2-core/ F: package/earlyoom/ F: package/gconf/ F: package/libnss/ +F: package/llama-cpp/ F: package/llvm-project/clang/ F: package/llvm-project/lld/ F: package/llvm-project/llvm/
package/Config.in +1 -0
@@ -2299,6 +2299,7 @@ comment "linux-pam plugins" source "package/libpam-tacplus/Config.in" endif source "package/liquid-dsp/Config.in" + source "package/llama-cpp/Config.in" source "package/llvm-project/llvm/Config.in" source "package/lttng-libust/Config.in" source "package/matio/Config.in"
package/llama-cpp/Config.in new file +36 -0
@@ -0,0 +1,36 @@ +config BR2_PACKAGE_LLAMA_CPP + bool "llama.cpp" + depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9 + depends on !BR2_s390x + depends on !BR2_riscv + help + LLM inference in C/C++ + +comment "llama-cpp needs glibc or musl toolchain w/ c++, gcc >= 9" + depends on !BR2_INSTALL_LIBSTDCPP \ + || !BR2_TOOLCHAIN_GCC_AT_LEAST_9 \ + || (!BR2_TOOLCHAIN_USES_GLIBC && !BR2_TOOLCHAIN_USES_MUSL) + +if BR2_PACKAGE_LLAMA_CPP + +config BR2_PACKAGE_LLAMA_CPP_TOOLS + bool "Enable tools" + help + Build CLI tools like llama-cli, llama-bench, etc. + +config BR2_PACKAGE_LLAMA_CPP_SERVER + bool "Enable server" + help + Build OpenAI API-compatible web server, llama-server. + +config BR2_PACKAGE_LLAMA_CPP_VULKAN + bool "Vulkan support" + depends on BR2_INSTALL_LIBSTDCPP # vulkan-loader + depends on !BR2_STATIC_LIBS # vulkan-loader + depends on BR2_TOOLCHAIN_HAS_THREADS # vulkan-loader + select BR2_PACKAGE_VULKAN_HEADERS + select BR2_PACKAGE_VULKAN_LOADER + help + Enable Vulkan backend for GPU acceleration. +endif
package/llama-cpp/llama-cpp.hash new file +4 -0
@@ -0,0 +1,4 @@ +# Locally calculated +sha256 35ad7f4e50252a7f8b4ead4d8ad7dd553f31ceb52c9ca325f2fcda0958e70336 b5369.tar.gz +# License +sha256 e562a2ddfaf8280537795ac5ecd34e3012b6582a147ef69ba6a6a5c08c84757d LICENSE
package/llama-cpp/llama-cpp.mk new file +43 -0
43 lines not on this page · raw message
---
base-commit: f23c810176305604c54b79611d885547f4e45c23
change-id: 20250618-llama-cpp-d3fc28b4562f
Best regards,
Signature · 6 lines
Joseph Kogut <[email protected]> _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot