package/llama-cpp: new package
PATCH v2 1/1 package/llama-cpp: new package
Add a package for llama.cpp, a C/C++ LLM inference library, used in popular projects like Ollama, RamaLama, and more.
---
I've bumped this package to the latest revision and I'm resending it for review as a v2.
This patch adds a package for llama.cpp, an LLM inference library, supporting many popular models, including LLaMa, Gemma, Deepseek, Qwen, and many more.
This library includes tools that can be used for standalone inference, like llama-cli and llama-server, as well as benchmarking, in llama-bench. The library has a variety of software and hardware accelerated backends, but this patch focuses on OpenBLAS and Vulkan support to start.
The patch disables building for S390X and RISCV due to compilation errors related to intrinsics, even though they're technically supported. Older GCC versions are also problematic, and GCC > v9 is required.
As mentioned in the virglrenderer series, this package was used to test virglrenderer with venus in a nested qemu guest using virtio-gpu.
https://lists.buildroot.org/pipermail/buildroot/2025-June/781395.html
The package was tested against all toolchains with utils/test-pkg:
40 builds, 28 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
Feedback welcome.
Thanks,
Joseph
---
Changes in v2:
- Bump version to b6818
- Link to v1: https://lore.kernel.org/r/[email protected]
--- DEVELOPERS | 1 + package/Config.in | 1 + package/llama-cpp/Config.in | 36 +++++++++++++++++++++++++++++++++ package/llama-cpp/llama-cpp.hash | 4 ++++ package/llama-cpp/llama-cpp.mk | 43 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 85 insertions(+)
DEVELOPERS +1 -0
@@ -1762,6 +1762,7 @@ F: package/at-spi2-core/ F: package/earlyoom/ F: package/gconf/ F: package/libnss/ +F: package/llama-cpp/ F: package/llvm-project/clang/ F: package/llvm-project/lld/ F: package/llvm-project/llvm/
package/Config.in +1 -0
@@ -2299,6 +2299,7 @@ comment "linux-pam plugins" source "package/libpam-tacplus/Config.in" endif source "package/liquid-dsp/Config.in" + source "package/llama-cpp/Config.in" source "package/llvm-project/llvm/Config.in" source "package/lttng-libust/Config.in" source "package/matio/Config.in"
package/llama-cpp/Config.in new file +36 -0
@@ -0,0 +1,36 @@ +config BR2_PACKAGE_LLAMA_CPP + bool "llama.cpp" + depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9 + depends on !BR2_s390x + depends on !BR2_riscv + help + LLM inference in C/C++ + +comment "llama-cpp needs glibc or musl toolchain w/ c++, gcc >= 9" + depends on !BR2_INSTALL_LIBSTDCPP \ + || !BR2_TOOLCHAIN_GCC_AT_LEAST_9 \ + || (!BR2_TOOLCHAIN_USES_GLIBC && !BR2_TOOLCHAIN_USES_MUSL) + +if BR2_PACKAGE_LLAMA_CPP + +config BR2_PACKAGE_LLAMA_CPP_TOOLS + bool "Enable tools" + help + Build CLI tools like llama-cli, llama-bench, etc. + +config BR2_PACKAGE_LLAMA_CPP_SERVER + bool "Enable server" + help + Build OpenAI API-compatible web server, llama-server. + +config BR2_PACKAGE_LLAMA_CPP_VULKAN + bool "Vulkan support" + depends on BR2_INSTALL_LIBSTDCPP # vulkan-loader + depends on !BR2_STATIC_LIBS # vulkan-loader + depends on BR2_TOOLCHAIN_HAS_THREADS # vulkan-loader + select BR2_PACKAGE_VULKAN_HEADERS + select BR2_PACKAGE_VULKAN_LOADER + help + Enable Vulkan backend for GPU acceleration. +endif
package/llama-cpp/llama-cpp.hash new file +4 -0
@@ -0,0 +1,4 @@ +# Locally calculated +sha256 44fb7508b578d7e263fb4c5ef8252f406cb5951a9891c107ee8553a5f1b2e186 b6818.tar.gz +# License +sha256 e562a2ddfaf8280537795ac5ecd34e3012b6582a147ef69ba6a6a5c08c84757d LICENSE
package/llama-cpp/llama-cpp.mk new file +43 -0
43 lines not on this page · raw message
---
base-commit: f23c810176305604c54b79611d885547f4e45c23
change-id: 20250618-llama-cpp-d3fc28b4562f
Best regards,
Signature · 6 lines
Joseph Kogut <[email protected]> _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hi Joseph,
Thanks for this patch. I have few comments, see below.
Quoted 5 lines · Joseph Kogut
Add a package for llama.cpp, a C/C++ LLM inference library, used in popular projects like Ollama, RamaLama, and more.
Signed-off-by Joseph Kogut <[email protected]>---
[...]
package/llama-cpp/Config.in
Quoted 6 lines
package/llama-cpp/Config.in new file +0
-0@@ -0,0 +1,36 @@
For restricting this package to some architectures, could you use the "standard" _ARCH_SUPPORTS pattern, at the beginning of the file, please? Like:
config BR2_PACKAGE_LLAMA_CPP_ARCH_SUPPORTS
bool default y depends on !BR2_s390x depends on !BR2_riscv
package/llama-cpp/Config.in line 3
Quoted 3 lines
package/llama-cpp/Config.in new file +3
-0+config BR2_PACKAGE_LLAMA_CPP + bool "llama.cpp" + depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL
I don't think this libc restriction is needed. This package could build correctly with uClibc too. It will just need to select libexecinfo when uclibc is used.
Then, the dependency on the newly introduced _ARCH_SUPPORTS needs to be added.
I think this package will likely also depends on having a toolchain with C++.
package/llama-cpp/Config.in line 6
Quoted 3 lines
package/llama-cpp/Config.in new file +3
-0+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9 + depends on !BR2_s390x + depends on !BR2_riscv
Those dependencies on architectures could be removed, with the _ARCH_SUPPORTS.
package/llama-cpp/Config.in line 8
Quoted 2 lines
package/llama-cpp/Config.in new file +2
-0+ help + LLM inference in C/C++
In Buildroot help strings, we usually add the upstream homepage url.
With those comments, the block should become:
config BR2_PACKAGE_LLAMA_CPP
bool "llama.cpp"
depends on BR2_PACKAGE_LLAMA_CPP_ARCH_SUPPORTS depends on BR2_INSTALL_LIBSTDCPP
depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9
select BR2_PACKAGE_LIBEXECINFO if BR2_TOOLCHAIN_USES_UCLIBC help
LLM inference in C/C++
package/llama-cpp/Config.in line 13
Quoted 5 lines
package/llama-cpp/Config.in new file +5
-0+ +comment "llama-cpp needs glibc or musl toolchain w/ c++, gcc >= 9" + depends on !BR2_INSTALL_LIBSTDCPP \ + || !BR2_TOOLCHAIN_GCC_AT_LEAST_9 \ + || (!BR2_TOOLCHAIN_USES_GLIBC && !BR2_TOOLCHAIN_USES_MUSL)
To keep a correct indentation in Kconfig, this comment should be moved at the end of the Config.in file. Please also adjust the condition accordingly (i.e. remove conditions on libc).
Quoted 6 lines
package/llama-cpp/Config.in new file +6
-0+ +if BR2_PACKAGE_LLAMA_CPP + +config BR2_PACKAGE_LLAMA_CPP_TOOLS + bool "Enable tools" + help
package/llama-cpp/Config.in line 29
Quoted 10 lines
package/llama-cpp/Config.in new file +10
-0+ Build CLI tools like llama-cli, llama-bench, etc. + +config BR2_PACKAGE_LLAMA_CPP_SERVER + bool "Enable server" + help + Build OpenAI API-compatible web server, llama-server. + +config BR2_PACKAGE_LLAMA_CPP_VULKAN + bool "Vulkan support" + depends on BR2_INSTALL_LIBSTDCPP # vulkan-loader
This dependency should not be needed here, since we'll have it in BR2_PACKAGE_LLAMA_CPP.
Quoted 7 lines
package/llama-cpp/Config.in new file +7
-0+ depends on !BR2_STATIC_LIBS # vulkan-loader + depends on BR2_TOOLCHAIN_HAS_THREADS # vulkan-loader + select BR2_PACKAGE_VULKAN_HEADERS + select BR2_PACKAGE_VULKAN_LOADER + help + Enable Vulkan backend for GPU acceleration. +endif
[...]
Quoted 16 lines
package/llama-cpp/llama-cpp.mk new file +9
-0@@ -0,0 +1,43 @@ +################################################################################ +# +# llama.cpp +# +################################################################################ + +LLAMA_CPP_VERSION = 6818 +LLAMA_CPP_SOURCE = b$(LLAMA_CPP_VERSION).tar.gz +LLAMA_CPP_SITE =
package/llama-cpp/llama-cpp.mk line 18
Quoted 9 lines
package/llama-cpp/llama-cpp.mk new file +9
-0+LLAMA_CPP_LICENSE = MIT +LLAMA_CPP_LICENSE_FILES = LICENSE +LLAMA_CPP_INSTALL_STAGING = YES +LLAMA_CPP_CONF_OPTS = \ + -DLLAMA_BUILD_TESTS=OFF \ + -DLLAMA_BUILD_EXAMPLES=OFF \ + -DLLAMA_FATAL_WARNINGS=OFF + +ifeq ($(BR2_PACKAGE_CURL),y)
In Buildroot, the symbol for cURL is BR2_PACKAGE_LIBCURL. So this line should become:
ifeq ($(BR2_PACKAGE_LIBCURL),y)
This can be detected by the check command:
utils/check-symbols
package/llama-cpp/llama-cpp.mk line 23
Quoted 5 lines
package/llama-cpp/llama-cpp.mk new file +5
-0+LLAMA_CPP_CONF_OPTS += -DLLAMA_CURL=ON +LLAMA_CPP_DEPENDENCIES += libcurl +else +LLAMA_CPP_CONF_OPTS += -DLLAMA_CURL=OFF +endif
Here, you could add the dependency for libexecinfo, to support uclibc.
ifeq ($(BR2_PACKAGE_LIBEXECINFO),y)LLAMA_CPP_DEPENDENCIES += libexecinfoLLAMA_CPP_LDFLAGS += -lexecinfo
endif
package/llama-cpp/llama-cpp.mk line 26
Quoted 3 lines
package/llama-cpp/llama-cpp.mk new file +3
-0+ +ifeq ($(BR2_PACKAGE_LLAMA_CPP_TOOLS),y) +LLAMA_CPP_CONF_OPTS += -DLLAMA_BUILD_TOOLS=ON
Here, you should handle the case which disable the tools, adding:
elseLAMA_CPP_CONF_OPTS += -DLLAMA_BUILD_TOOLS=OFF
package/llama-cpp/llama-cpp.mk line 30
Quoted 4 lines
package/llama-cpp/llama-cpp.mk new file +4
-0+endif + +ifeq ($(BR2_PACKAGE_LLAMA_CPP_SERVER),y) +LLAMA_CPP_CONF_OPTS += -DLLAMA_BUILD_SERVER=ON
Same comment for the _SERVER, adding:
elseLLAMA_CPP_CONF_OPTS += -DLLAMA_BUILD_SERVER=OFF
package/llama-cpp/llama-cpp.mk line 35
Quoted 5 lines
package/llama-cpp/llama-cpp.mk new file +5
-0+endif + +ifeq ($(BR2_PACKAGE_LLAMA_CPP_VULKAN),y) +LLAMA_CPP_DEPENDENCIES += vulkan-loader +LLAMA_CPP_CONF_OPTS += -DGGML_VULKAN=ON
Same comment. Adding here:
elseLLAMA_CPP_CONF_OPTS += -DGGML_VULKAN=OFF
package/llama-cpp/llama-cpp.mk line 40
Quoted 5 lines
package/llama-cpp/llama-cpp.mk new file +5
-0+endif + +ifeq ($(BR2_PACKAGE_OPENBLAS),y) +LLAMA_CPP_DEPENDENCIES += openblas +LLAMA_CPP_CONF_OPTS += -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
Same comment. Adding here:
elseLLAMA_CPP_CONF_OPTS += -DGGML_BLAS=OFF
Quoted 1 line
+endif
Finally, for handling the libexecinfo for uclibc, we need to pass the ldflags:
LLAMA_CPP_CONF_OPTS += \
-DCMAKE_EXE_LINKER_FLAGS="$(LLAMA_CPP_LDFLAGS)"Quoted 10 lines
+
+$(eval $(cmake-package))---
base-commit:f23c810176305604c54b79611d885547f4e45c23
change-id: 20250618-llama-cpp-d3fc28b4562fBest regards,
--
Joseph Kogut <[email protected]>
Could you send an updated patch including those changes, please?
Best regards,
Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Hello Julien,
Quoted 211 lines · Julien Olivain
Hi Joseph,
Thanks for this patch. I have few comments, see below.
Quoted 5 lines · Joseph Kogut
Add a package for llama.cpp, a C/C++ LLM inference library, used in popular projects like Ollama, RamaLama, and more.
Signed-off-by Joseph Kogut <[email protected]>---
[...]
package/llama-cpp/Config.in
Quoted 6 lines
package/llama-cpp/Config.in new file +0
-0@@ -0,0 +1,36 @@For restricting this package to some architectures, could you use the "standard"
_ARCH_SUPPORTSpattern, at the beginning of the file, please? Like:config
BR2_PACKAGE_LLAMA_CPP_ARCH_SUPPORTS
bool
default y
depends on !BR2_s390x
depends on !BR2_riscvpackage/llama-cpp/Config.in line 3
Quoted 3 lines
package/llama-cpp/Config.in new file +3
-0+config BR2_PACKAGE_LLAMA_CPP + bool "llama.cpp" + depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSLI don't think this libc restriction is needed. This package could build correctly with uClibc too. It will just need to select libexecinfo when uclibc is used.
Then, the dependency on the newly introduced
_ARCH_SUPPORTSneeds to be added.I think this package will likely also depends on having a toolchain with C++.
package/llama-cpp/Config.in line 6
Quoted 3 lines
package/llama-cpp/Config.in new file +3
-0+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9 + depends on !BR2_s390x + depends on !BR2_riscvThose dependencies on architectures could be removed, with the
_ARCH_SUPPORTS.package/llama-cpp/Config.in line 8
Quoted 2 lines
package/llama-cpp/Config.in new file +2
-0+ help + LLM inference in C/C++In Buildroot help strings, we usually add the upstream homepage url.
With those comments, the block should become:
config
BR2_PACKAGE_LLAMA_CPP
bool "llama.cpp"
depends onBR2_PACKAGE_LLAMA_CPP_ARCH_SUPPORTSdepends onBR2_INSTALL_LIBSTDCPP
depends onBR2_TOOLCHAIN_GCC_AT_LEAST_9
selectBR2_PACKAGE_LIBEXECINFOifBR2_TOOLCHAIN_USES_UCLIBChelp
LLM inference in C/C++package/llama-cpp/Config.in line 13
Quoted 5 lines
package/llama-cpp/Config.in new file +5
-0+ +comment "llama-cpp needs glibc or musl toolchain w/ c++, gcc >= 9" + depends on !BR2_INSTALL_LIBSTDCPP \ + || !BR2_TOOLCHAIN_GCC_AT_LEAST_9 \ + || (!BR2_TOOLCHAIN_USES_GLIBC && !BR2_TOOLCHAIN_USES_MUSL)To keep a correct indentation in
Kconfig, this comment should be moved at the end of the Config.in file. Please also adjust the condition accordingly (i.e. remove conditions on libc).Quoted 6 lines
package/llama-cpp/Config.in new file +6
-0+ +if BR2_PACKAGE_LLAMA_CPP + +config BR2_PACKAGE_LLAMA_CPP_TOOLS + bool "Enable tools" + helppackage/llama-cpp/Config.in line 29
Quoted 10 lines
package/llama-cpp/Config.in new file +10
-0+ Build CLI tools like llama-cli, llama-bench, etc. + +config BR2_PACKAGE_LLAMA_CPP_SERVER + bool "Enable server" + help + Build OpenAI API-compatible web server, llama-server. + +config BR2_PACKAGE_LLAMA_CPP_VULKAN + bool "Vulkan support" + depends on BR2_INSTALL_LIBSTDCPP # vulkan-loaderThis dependency should not be needed here, since we'll have it in
BR2_PACKAGE_LLAMA_CPP.Quoted 7 lines
package/llama-cpp/Config.in new file +7
-0+ depends on !BR2_STATIC_LIBS # vulkan-loader + depends on BR2_TOOLCHAIN_HAS_THREADS # vulkan-loader + select BR2_PACKAGE_VULKAN_HEADERS + select BR2_PACKAGE_VULKAN_LOADER + help + Enable Vulkan backend for GPU acceleration. +endif[...]
Quoted 17 lines
package/llama-cpp/llama-cpp.mk new file +9
-0@@ -0,0 +1,43 @@ +################################################################################ +# +# llama.cpp +# +################################################################################ + +LLAMA_CPP_VERSION = 6818 +LLAMA_CPP_SOURCE = b$(LLAMA_CPP_VERSION).tar.gz +LLAMA_CPP_SITE =package/llama-cpp/llama-cpp.mk line 18
Quoted 9 lines
package/llama-cpp/llama-cpp.mk new file +9
-0+LLAMA_CPP_LICENSE = MIT +LLAMA_CPP_LICENSE_FILES = LICENSE +LLAMA_CPP_INSTALL_STAGING = YES +LLAMA_CPP_CONF_OPTS = \ + -DLLAMA_BUILD_TESTS=OFF \ + -DLLAMA_BUILD_EXAMPLES=OFF \ + -DLLAMA_FATAL_WARNINGS=OFF + +ifeq ($(BR2_PACKAGE_CURL),y)In Buildroot, the symbol for cURL is
BR2_PACKAGE_LIBCURL. So this line should become:ifeq ($(BR2_PACKAGE_LIBCURL),y)
This can be detected by the check command:
utils/check-symbolspackage/llama-cpp/llama-cpp.mk line 23
Quoted 5 lines
package/llama-cpp/llama-cpp.mk new file +5
-0+LLAMA_CPP_CONF_OPTS += -DLLAMA_CURL=ON +LLAMA_CPP_DEPENDENCIES += libcurl +else +LLAMA_CPP_CONF_OPTS += -DLLAMA_CURL=OFF +endifHere, you could add the dependency for libexecinfo, to support uclibc.
ifeq ($(BR2_PACKAGE_LIBEXECINFO),y)
LLAMA_CPP_DEPENDENCIES+= libexecinfoLLAMA_CPP_LDFLAGS+= -lexecinfo
endifpackage/llama-cpp/llama-cpp.mk line 26
Quoted 3 lines
package/llama-cpp/llama-cpp.mk new file +3
-0+ +ifeq ($(BR2_PACKAGE_LLAMA_CPP_TOOLS),y) +LLAMA_CPP_CONF_OPTS += -DLLAMA_BUILD_TOOLS=ONHere, you should handle the case which disable the tools, adding:
else
LAMA_CPP_CONF_OPTS+= -DLLAMA_BUILD_TOOLS=OFFpackage/llama-cpp/llama-cpp.mk line 30
Quoted 4 lines
package/llama-cpp/llama-cpp.mk new file +4
-0+endif + +ifeq ($(BR2_PACKAGE_LLAMA_CPP_SERVER),y) +LLAMA_CPP_CONF_OPTS += -DLLAMA_BUILD_SERVER=ONSame comment for the
_SERVER, adding:else
LLAMA_CPP_CONF_OPTS+= -DLLAMA_BUILD_SERVER=OFFpackage/llama-cpp/llama-cpp.mk line 35
Quoted 5 lines
package/llama-cpp/llama-cpp.mk new file +5
-0+endif + +ifeq ($(BR2_PACKAGE_LLAMA_CPP_VULKAN),y) +LLAMA_CPP_DEPENDENCIES += vulkan-loader +LLAMA_CPP_CONF_OPTS += -DGGML_VULKAN=ONSame comment. Adding here:
else
LLAMA_CPP_CONF_OPTS+= -DGGML_VULKAN=OFFpackage/llama-cpp/llama-cpp.mk line 40
Quoted 5 lines
package/llama-cpp/llama-cpp.mk new file +5
-0+endif + +ifeq ($(BR2_PACKAGE_OPENBLAS),y) +LLAMA_CPP_DEPENDENCIES += openblas +LLAMA_CPP_CONF_OPTS += -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLASSame comment. Adding here:
else
LLAMA_CPP_CONF_OPTS+= -DGGML_BLAS=OFFQuoted 1 line
+endif
Finally, for handling the libexecinfo for uclibc, we need to pass the ldflags:
LLAMA_CPP_CONF_OPTS += \ -DCMAKE_EXE_LINKER_FLAGS="$(LLAMA_CPP_LDFLAGS)"Quoted 7 lines
+
+$(eval $(cmake-package))---
base-commit:f23c810176305604c54b79611d885547f4e45c23
change-id: 20250618-llama-cpp-d3fc28b4562f
Quoted 6 lines
Quoted 3 lines
Best regards,
--
Joseph Kogut <[email protected]>Could you send an updated patch including those changes, please?
Absolutely, thank you for taking the time to give such a thorough and well explained review. I'll follow up.
Quoted 3 lines
Best regards,
Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot