Cover Image for Using llama.cpp to self-host Large Language Models in Production

A practical guide to self-hosting LLMs in production using llama.cpp's llama-server with Docker compose and Systemd

Author Demis Bellot

More from the blog

Article

Tailwind CSS v4.0 is here!

All ASP .NET Core Identity Auth Tailwind Templates upgraded to Tailwind CSS v4
Demis Bellot Tailwind CSS v4.0 is here! background

Demis Bellot

21 min read

Article

New Angular 19 ASP.NET Core Identity Auth Tailwind SPA Template

Introducing our new Angular 19 IdentityAuth Tailwind Template with DarkMode and support for RxJS and Signals dev models
Demis Bellot New Angular 19 ASP.NET Core Identity Auth Tailwind SPA Template background

Demis Bellot

9 min read

Article

AI Server - May 2025 Update

New SOTA LLMs added, support for thinking responses, Ollama Vision Models & Generate API
Demis Bellot AI Server - May 2025 Update background

Demis Bellot

5 min read

Article

Text to Blazor upgraded to use best coding LLMs

Text to Blazor now generates CRUD Apps using best coding models from Google, Anthropic, Meta, DeepSeek & Mistral
Demis Bellot Text to Blazor upgraded to use best coding LLMs background

Demis Bellot

5 min read

Article

Complete Admin UI Management for Identity Claims and Roles

With support for adding Roles and User and Role Claims, the built-in Admin UI now offers complete Identity Auth management
Demis Bellot Complete Admin UI Management for Identity Claims and Roles background

Demis Bellot

2 min read

Article

In Depth and Interactive API Analytics available to all ServiceStack Apps

Comprehensive and Interactive API Analytics in Admin UI offering deep insights into API usage, performance, users, API Keys & IPs.
Demis Bellot In Depth and Interactive API Analytics available to all ServiceStack Apps background

Demis Bellot

7 min read

llms.txt and llms-full.txt support for Razor SSG and Razor Press

Free Razor SSG and Razor Press Static Site Templates now auto generates llms.txt and llms-full.txt files for Large Language Models
Demis Bellot llms.txt and llms-full.txt support for Razor SSG and Razor Press background

Demis Bellot

4 min read

Generate CRUD APIs and Admin UIs from existing DBs

Use the metadata from an existing RDBMS to generate AutoQuery CRUD APIs and Admin UIs for managing your data.
Demis Bellot Generate CRUD APIs and Admin UIs from existing DBs background

Demis Bellot

3 min read

FREE CLI Access to the worlds most popular AI Models

No API Keys, no Signups, just free access to the worlds most popular AI Large Language Models
Demis Bellot FREE CLI Access to the worlds most popular AI Models background

Demis Bellot

3 min read

New okai tool for Rapid App Development

Explore the new okai tool and workflow for rapidly developing Blazor Vue Crud Apps with AI and TypeScript Data Models
Demis Bellot New okai tool for Rapid App Development background

Demis Bellot

3 min read

Text to Blazor Vue CRUD Apps

Discover how to quickly generate Blazor Admin CRUD Apps from a text description using the new okai tool
Demis Bellot Text to Blazor Vue CRUD Apps background

Demis Bellot

14 min read

Self Hosted AI Server for LLMs, Ollama, Comfy UI & FFmpeg

Introducing AI Server - an OSS Self Hosted Gateway for running LLM, Ollama, Media and Comfy UI APIs
Demis Bellot Self Hosted AI Server for LLMs, Ollama, Comfy UI & FFmpeg background

Demis Bellot

8 min read

.NET 8 Templates migrated to use Kamal for deployments

All Identity Auth templates now use Kamal for deployments, simplifying the process of deploying .NET Apps to any Linux server.
Darren Reid .NET 8 Templates migrated to use Kamal for deployments background

Darren Reid

5 min read

DTOs in all languages downloadable without .NET

ServiceStack Typed DTOs for all languages can now be downloaded and uploaded without .NET installed
Demis Bellot DTOs in all languages downloadable without .NET background

Demis Bellot

4 min read

ServiceStack.Swift client library rewritten for Swift 6

ServiceStack.Swift has been rewritten to take advantage of Swift 6 features, now dependency-free.
Demis Bellot ServiceStack.Swift client library rewritten for Swift 6 background

Demis Bellot

3 min read

Typed Open AI Chat & Ollama APIs in 11 Languages

Introducing AI Server - an OSS Self Hosted Gateway for running LLM, Ollama, Media and Comfy UI APIs
Demis Bellot Typed Open AI Chat & Ollama APIs in 11 Languages background

Demis Bellot

5 min read

Simple API Keys Credentials Provider for .NET 8 C# Microservices

Improved Simple Auth using API Keys and Admin UI that's ideal for .NET 8 C# Microservices
Demis Bellot Simple API Keys Credentials Provider for .NET 8 C# Microservices background

Demis Bellot

4 min read

Podcasts now in Razor SSG

Razor SSG now includes support for Podcasts
Demis Bellot Podcasts now in Razor SSG background

Demis Bellot

5 min read

Scalable Server SQLite Apps

Learn how to build concurrent, safe scalable SQLite .NET Apps in C#
Demis Bellot Scalable Server SQLite Apps background

Demis Bellot

10 min read

SQLite C# Request Logs

Maintain C# Request Log History in rolling Monthly SQLite databases
Demis Bellot SQLite C# Request Logs background

Demis Bellot

3 min read

Simple C# Background Jobs & Recurring Tasks for .NET 8

Introducing C# Background Jobs for managing background jobs and scheduled tasks
Demis Bellot Simple C# Background Jobs & Recurring Tasks for .NET 8 background

Demis Bellot

20 min read
view all posts
MyApp logo

ServiceStack started development in 2008 with the mission of creating a best-practices services framework with an emphasis on simplicity and speed

more information