WORK IN PROGRESS

ISSUES WITH VPC ACCESS and ec2 api service

Remote AI Inference for Coding

This project runs coding-focused AI models remotely so your local machine doesn’t melt.

Core goal: provide a secure, private, low-cost, on-demand AI inference backend that your IDE can talk to over HTTPS, without exposing anything publicly or paying for idle GPUs.

What this project is (opinionated)

• A personal AI coding backend (think: remote Copilot-style brain, but yours)
• Fully managed via Infrastructure as Code (Terraform)
• Private-network only (no public IPs, no domains, no internet exposure)
• Optimized for cost control: scale to zero, auto-scale on demand
• Cloud-agnostic by design, starting with AWS

What this project is NOT

• Not a SaaS
• Not multi-tenant
• Not a chatbot UI
• Not a long-running always-on GPU box

Typical workflow

1. Your IDE sends coding requests over HTTPS
2. Traffic goes through a private VPN into your VPC
3. Requests hit an autoscaled AI inference service
4. When idle → infrastructure scales down → you pay almost nothing

Why this exists

Local AI models are great — until:

• Fans go to 100%
• Battery dies
• Laptop thermal-throttles
• RAM gets obliterated

This project moves the heavy lifting to the cloud without giving up privacy, security, or cost control.

---

Prerequisites

• AWS account with permissions for:

  • EC2, Auto Scaling, IAM
  • ACM (certificate import)
  • Client VPN

• Terraform >= 1.5

• AWS CLI configured

• OpenSSL installed locally

---

IAM Permissions Required for Certificate Import

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "acm:ImportCertificate",
                "acm:ListCertificates",
                "acm:DescribeCertificate"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "*"
        }
    ]
}

---

🔐 Client VPN Certificates (no real domain)

AWS Client VPN requires the server certificate to have a CN or SAN resembling a domain, with proper key usage. We use a self-signed certificate with a dummy domain.

Step 1 — Terraform: deploy infrastructure

terraform init
terraform validate
terraform plan
terraform apply

After terraform apply, note your VPC ID, subnets, security groups, and VPN endpoint ARN.

Step 2 — Create a private CA

mkdir vpn-certs && cd vpn-certs
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 -out ca.crt -subj "/CN=personal-llm-ca"

Step 3 — Create VPN server certificate

cat > server.ext <<EOF
basicConstraints=CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = DNS:llm-vpn.local
EOF

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/CN=llm-vpn.local"
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365 -sha256 -extfile server.ext

Step 4 — Import server cert into ACM

aws acm import-certificate --certificate fileb://server.crt --private-key fileb://server.key --certificate-chain fileb://ca.crt

• Save the returned ARN into terraform.tfvars as vpn_server_cert_arn
• Re-apply Terraform if needed.

Step 5 — Get VPN endpoint details

aws ec2 describe-client-vpn-endpoints --query 'ClientVpnEndpoints[*].[ClientVpnEndpointId,DnsName]' --output table

• Note DnsName or private IP for your VPN endpoint.

Step 6 — Create a client certificate

openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr -subj "/CN=personal-llm-client"
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 365 -sha256

Step 7 — Generate client .ovpn config

cat > client.ovpn <<EOF
client
dev tun
proto udp
remote <VPN_ENDPOINT_DNS_OR_IP> 443
resolv-retry infinite
nobind
persist-key
persist-tun
<ca>
$(cat ca.crt)
</ca>
<cert>
$(cat client.crt)
</cert>
<key>
$(cat client.key)
</key>
remote-cert-tls server
cipher AES-256-CBC
verb 3
EOF

• Replace <VPN_ENDPOINT_DNS_OR_IP> with the value from Step 5.

Step 8 — Connect VPN

/opt/homebrew/opt/openvpn/sbin/openvpn --config client.ovpn

• Verify connection: ping <LLM_backend_private_IP>

---

IDE Integration & LLM Verification

1. Get LLM Backend Private IP

aws ec2 describe-instances \
  --filters "Name=instance-state-name,Values=running" \
  --query "Reservations[].Instances[].PrivateIpAddress" \
  --output text

• Use this IP for IDE config or direct tests.

• If you can't reach the LLM after connecting to the VPN, verify routing/authorization and security group rules. Prefer making changes through Terraform (IaC) to avoid state drift — run terraform init, terraform plan, terraform apply, or terraform destroy as appropriate.

Terraform outputs include suggested commands for targeted changes like VPN routes, authorization rules, and security-group updates; run terraform init then terraform output -json to view them in your environment.

2. Verify LLM Models Endpoint

curl -k https://<PRIVATE_IP>:443/v1/models

• Should return available AI models.

3. IDE Configuration (Continue / Ollama)

name: Local Config
version: 1.0.0
schema: v1
models:
  - name: Llama 3.1 8B
    provider: ollama
    model: llama3.1:8b
    roles:
      - chat
      - edit
      - apply
  - name: Qwen2.5-Coder 1.5B
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles:
      - autocomplete
  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text:latest
    roles:
      - embed

• Use private IP from Step 1 if endpoint override is needed.

---

Cost & Billing Check

aws ce get-cost-and-usage \
    --time-period Start=$(date -u -v-30d "+%Y-%m-%d"),End=$(date -u "+%Y-%m-%d") \
    --granularity MONTHLY \
    --metrics "UnblendedCost" \
    --group-by Type=DIMENSION,Key=SERVICE