Using llama.cpp to self-host Large Language Models in Production
A practical guide to self-hosting LLMs in production using llama.cpp's llama-server with Docker compose and Systemd
Backstage pass to all things ServiceStack & .NET
Listen
Episodes
ai
release
admin-ui
api
.net8
vue
autoquery
gpt
identity-auth
okai
ai-server
service-reference
blazor
js
markdown
auth
apikeys
commands
template
angular
react
c#
sqlite
jobs
templates
ssg
docker
razor
logging
kotlin
tailwind
analytics
apis
github-actions
devops
swift
performance
scalability
rdbms
caching
redis
middleware
ios
android
vite
docs
ioc
json
openapi
swagger
mvc
AI Server - May 2025 Update
New SOTA LLMs added, support for thinking responses, Ollama Vision Models & Generate API
Introducing AI Server
Introducing AI Server - an OSS Self Hosted Gateway for running LLM, Ollama, Media and Comfy UI APIs
Typed Open AI Chat & Ollama APIs in 11 Languages
Introducing AI Server - an OSS Self Hosted Gateway for running LLM, Ollama, Media and Comfy UI APIs
ServiceStack v8.5 Release
The latest features in ServiceStack v8.5 covering release of AI Server and Kamal Deployments and more!