NG: llamacpp on the ai node (Qwen3.8-27B) with the AMD GPU device plugin
Update Kubernetes Services Wiki / Generate and Update K8s Wiki (push) Successful in 44s

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Aleksandr BogomiakovandClaude committed 2026-10-06 19:20:06 +01:00
1 parent bd88daa109
commit 373ce77ac8
7 files changed
+169

No files matched your search

+17
View File
@@ -0,0 +1,17 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: llamacpp-config
data:
LLAMA_CACHE: /models
LLAMA_ARG_HOST: 0.0.0.0
LLAMA_ARG_PORT: "8080"
# The model is chosen here: <hf-user>/<repo>:<quant>. It is downloaded into
# /models on first start; changing it restarts the pod (reloader).
LLAMA_ARG_HF_REPO: "unsloth/Qwen3.8-27B-GGUF:UD-Q6_K"
LLAMA_ARG_CTX_SIZE: "131072"
LLAMA_ARG_N_GPU_LAYERS: all
LLAMA_ARG_FLASH_ATTN: "on"
LLAMA_ARG_FIT: "on"
# Strix Halo: mmap-ed weights hang or corrupt on ROCm; read them directly.
LLAMA_ARG_LOAD_MODE: dio