Kyllene
Collection
Model series for RPG/ERP • 4 items • Updated • 4
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TeeZee/Kyllene-57B-v1.0-GGUF:# Run inference directly in the terminal:
llama cli -hf TeeZee/Kyllene-57B-v1.0-GGUF:Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf TeeZee/Kyllene-57B-v1.0-GGUF:# Run inference directly in the terminal:
./build/bin/llama cli -hf TeeZee/Kyllene-57B-v1.0-GGUF:Use Docker
docker model run hf.co/TeeZee/Kyllene-57B-v1.0-GGUF:Quick Links
GGUF quants of TeeZee/Kyllene-57B-v1.0, remeber to set your max context length to proper length for your hardware, 4096 is fine. Default context length is 200k so it will eat RAM or VRAM like crazy if left unchecked.
- Downloads last month
- 299
Hardware compatibility
Log In to add your hardware
2-bit
3-bit
4-bit
5-bit
6-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf TeeZee/Kyllene-57B-v1.0-GGUF:# Run inference directly in the terminal: llama cli -hf TeeZee/Kyllene-57B-v1.0-GGUF: