How to use LM Studio, KoboldCpp, llama.cpp or Ollama with Thaluna
Thaluna can use external local AI backends through its Local API / OpenAI-compatible mode.
This means you can run your own model locally with tools like LM Studio, KoboldCpp, llama.cpp server, or Ollama, then connect Thaluna to it.
Full setup guide:
https://thaluna.app/local-api-guide
What this is useful for
Better translation quality than the built-in lightweight offline models
Fully local translation without using cloud APIs
Using your own GGUF models
Lower latency if your local server is configured well
More control over model size, quantization, context, GPU usage, and speed
Recommended local options
LM Studio
Best if you want an easy GUI setup.
Load a GGUF model, start the local server, then connect Thaluna to the LM Studio API endpoint.
KoboldCpp
Good for advanced local setups and GGUF models.
It can be faster or more configurable than simpler setups depending on your GPU and settings.
llama.cpp server
Good for users who want direct control over llama.cpp flags, context size, GPU layers, and performance tuning.
Ollama
Easiest local setup for many users.
Recommended if you want something simple, but it may not always be the fastest option.
Basic setup idea
1. Start your local model server in LM Studio, KoboldCpp, llama.cpp, or Ollama.
2. Open Thaluna.
4. Go to Ollama/Cloud Setting
4. Enter the local API endpoint URL.
5. Enter the model name expected by your server.
6. Select Local API: OpenAI Compatible as the translation engine.
7. Set the target language and start translating.
Common local API endpoints
LM Studio
Endpoint:
http://localhost:1234/v1
KoboldCpp
OpenAI-compatible endpoint:
http://localhost:5001/v1
This may be different if you changed the KoboldCpp port or server settings.
llama.cpp server
Common endpoint:
http://localhost:8080/v1
This depends on the launch flags you used.
Ollama
Use Thaluna’s built-in Ollama integration if possible.
If you are using an OpenAI-compatible bridge for Ollama, use the endpoint provided by that bridge.
Exact setup details:
https://thaluna.app/local-api-guide
Important notes
If nothing appears at first, your local model may still be loading. Some backends load the model only when the first request arrives.
Use a smaller capture area in Thaluna when translating fast game dialogue.
Huge context sizes are usually unnecessary for OCR translation and can waste VRAM.
A smaller fast model can feel better for live gameplay than a larger slow model.
If you use RTX 5000 series GPUs and GPU OCR or built-in GPU models hang, set OCR / Translation Device in Thaluna to CPU and use an external backend like KoboldCpp, LM Studio, llama.cpp, Ollama, or OpenRouter.
Related guides
Local API / OpenAI-compatible setup guide
https://thaluna.app/local-api-guide
Translation quality and model recommendation guide
https://thaluna.app/translation-quality-guide
OpenRouter setup guide
https://thaluna.app/openrouter-guide
Hardware compatibility guide
https://thaluna.app/hardware-compatibility
Not sure which model to use?
Start with the translation quality guide first: