Using llama.cpp to self-host Large Language Models in Production
A practical guide to self-hosting LLMs in production using llama.cpp's llama-server with Docker compose and Systemd
Backstage pass to all things ServiceStack & .NET
Listen
Episodes
ai
release
admin-ui
api
.net8
vue
autoquery
gpt
identity-auth
okai
ai-server
service-reference
blazor
js
markdown
auth
apikeys
commands
template
angular
react
c#
sqlite
jobs
templates
ssg
docker
razor
logging
kotlin
tailwind
analytics
apis
github-actions
devops
swift
performance
scalability
rdbms
caching
redis
middleware
ios
android
vite
docs
ioc
json
openapi
swagger
mvc
AI Server - May 2025 Update
New SOTA LLMs added, support for thinking responses, Ollama Vision Models & Generate API
llms.txt and llms-full.txt support for Razor SSG and Razor Press
Free Razor SSG and Razor Press Static Site Templates now auto generates llms.txt and llms-full.txt files for Large Language Models
FREE Access to the worlds most popular AI Models
No API Keys, no Signups, just free access to the worlds most popular AI Large Language Models
Introducing AI Server
Introducing AI Server - an OSS Self Hosted Gateway for running LLM, Ollama, Media and Comfy UI APIs
Typed Open AI Chat & Ollama APIs in 11 Languages
Introducing AI Server - an OSS Self Hosted Gateway for running LLM, Ollama, Media and Comfy UI APIs