Self-Hosted LLM Inference Server (macOS + Tailscale)

A private AI inference server running open-weight models (DeepSeek R1, Qwen3, GPT-OSS) on Apple Silicon, exposed securely to my personal devices over an encrypted mesh network — at $0/month, with no data leaving my own hardware.

Stack: Ollama (Homebrew/launchd service) · Tailscale (WireGuard mesh VPN) · macOS (M4 Pro, 24GB unified memory) · deepseek-r1:14b / qwen3-coder:30b / gpt-oss:20b

---

Architecture

[iPhone on cellular]                    [MacBook M4 Pro]
       │                                      │
       │   Tailscale encrypted tunnel         │
       └────────► tailnet (100.x/8) ◄─────────┘
                        │
                        ▼
          http://100.121.112.127:11434
                        │
                        ▼
            Ollama (launchd service)
            bound to 0.0.0.0:11434
                        │
                        ▼
         deepseek-r1:14b (9GB, local)

No ports are exposed to the public internet. Only devices authenticated to my Tailscale tailnet can reach the endpoint. Traffic is end-to-end encrypted via WireGuard.

Why this project

• Demonstrate the full self-hosted inference stack: model serving, service management, private networking, API testing
• Keep sensitive workloads (unreleased music metadata, client work) entirely on my own hardware
• Establish a baseline architecture to replicate on cloud infrastructure (OCI/AWS) — see Next steps

Setup

1. Private network (Tailscale)

Installed Tailscale on the Mac (server) and iPhone (client), authenticated both to the same tailnet.

brew install --cask tailscale
tailscale ip -4
# 100.121.112.127

2. Bind Ollama to the network

By default Ollama listens on localhost only:

COMMAND  PID         USER   FD   TYPE   NODE NAME
ollama  4463 shadrickcarr    3u  IPv4    TCP localhost:11434 (LISTEN)

Set the bind address and restart the service:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
brew services restart ollama

Verified the new binding:

COMMAND   PID         USER   FD   TYPE   NODE NAME
ollama  19813 shadrickcarr    3u  IPv6    TCP *:11434 (LISTEN)

*:11434 = listening on all interfaces, reachable from the tailnet.

3. Models

ollama list
# qwen3-coder:30b     18 GB   — coding workhorse
# gpt-oss:20b         13 GB   — general purpose
# deepseek-r1:14b     9.0 GB  — reasoning (used for benchmarks below)
# nomic-embed-text    274 MB  — embeddings only

Verification

API test over the tailnet

curl http://100.121.112.127:11434/api/generate -d '{
  "model": "deepseek-r1:14b",
  "prompt": "Explain what a VPC is in two sentences.",
  "stream": false
}'

Response (trimmed):

{
  "model": "deepseek-r1:14b",
  "response": "A Virtual Private Cloud (VPC) is a secure, isolated network environment offered by AWS that allows you to host resources in the cloud. It provides full control over networking configurations, enabling customization of IP addressing, subnets, route tables, and security groups...",
  "done_reason": "stop",
  "total_duration": 47668794250,
  "load_duration": 31056803666,
  "eval_count": 375,
  "eval_duration": 15901575417
}

Remote client test

From an iPhone on cellular only (Wi-Fi disabled), connected via the Tailscale app:

http://100.121.112.127:11434 → "Ollama is running"

Note the 5G indicator in the status bar — the request traversed the cellular network and Tailscale tunnel, not local Wi-Fi. The browser's "Not Secure" label reflects the plain-HTTP endpoint; transport is nonetheless encrypted end-to-end by Tailscale's WireGuard tunnel. Terminating TLS properly (Caddy reverse proxy) is listed in next steps.

Full terminal session — service diagnosis, rebinding, model inventory, and the API test:

Benchmarks (deepseek-r1:14b on M4 Pro)

Metric	Value
Cold-start model load (9GB)	~31 s
Prompt eval	14 tokens / 0.55 s
Generation throughput	375 tokens / 15.9 s ≈ 23.6 tok/s
Total request (cold)	~47.7 s
Total request (warm)	~16 s

Note: R1 is a reasoning model — the API returns a separate thinking field with chain-of-thought before the final response. Downstream consumers (e.g. n8n workflows) should parse response only.

Troubleshooting log

Symptom: launchctl setenv OLLAMA_HOST had no effect; lsof kept showing localhost:11434 and there was no Ollama menu bar icon to restart.

Diagnosis: ps -p <PID> -o comm= revealed the running binary was /opt/homebrew/opt/ollama/bin/ollama — a Homebrew-managed launchd service, not the Ollama desktop app. The env var had been set, but the service needed a restart to pick it up.

Fix: brew services restart ollama, then re-verified with lsof -iTCP:11434 -sTCP:LISTEN. (Fallback if the service hadn't inherited the variable: add an EnvironmentVariables dict to ~/Library/LaunchAgents/homebrew.mxcl.ollama.plist — the macOS equivalent of editing a systemd unit file.)

Lesson: always identify how a service is managed (app vs. launchd/systemd) before assuming how to configure it.

Security considerations

• Access control: endpoint reachable only from devices authenticated to my tailnet; nothing exposed to the public internet. Tailscale handles key rotation and device auth.
• Known tradeoff: binding to 0.0.0.0 also answers on the local LAN, not just the tailnet. Acceptable on a trusted home network; on untrusted Wi-Fi, mitigate with the macOS firewall or by binding to the Tailscale interface IP specifically.
• Data sovereignty: prompts and outputs never leave my hardware — relevant for unreleased creative work and client material, and the reason to run DeepSeek as local open weights rather than via its hosted service.

Next steps

• Replicate on Oracle Cloud Always Free (ARM VM, 4 OCPU / 24GB): Ollama as a systemd service, OCI security lists in place of Tailscale-only access
• Short AWS EC2 deployment run to document the same architecture with security groups + IAM
• Point n8n workflows at this endpoint for local caption/metadata generation
• Add an HTTPS reverse proxy (Caddy) and basic auth as a production-hardening exercise