Gemma on Azure Container Apps

Deploy Google's Gemma 4 model as a secure, GPU-accelerated API endpoint on Azure Container Apps — using Ollama as the inference runtime, fronted by Nginx with HTTP Basic Auth.

Architecture

Internet → Azure Container Apps (HTTPS)
              └── Nginx (Basic Auth) → Ollama (Gemma 4)

• Ollama serves the model on port 11434
• Nginx handles authentication and proxies all traffic on port 80
• Azure Container Apps provides the public HTTPS endpoint and GPU-backed compute
• The model is pulled at container startup (not baked into the image), keeping the image small (~1 GB)

Prerequisites

• Azure CLI with the containerapp extension

  az extension add --name containerapp

• An Azure subscription with access to GPU workload profiles
• Logged in to Azure CLI: az login

GPU Availability

The Consumption-GPU-NC8as-T4 profile (1× NVIDIA T4, 16 GB VRAM) is available in these regions:

Region	T4	A100
eastus	✓	✓
westus	✓	✓
westus2	✓
westus3	✓	✓
westeurope	✓
southeastasia	✓
australiaeast	✓	✓

The default region in deploy.sh is eastus.

Deploy

chmod +x deploy.sh
./deploy.sh

The script will prompt for:

Prompt	Default	Description
Docker Hub image	rishabkumar7/gemma-ollama-auth:latest	Public image to deploy
Resource group name	rg-gemma-aca	Azure resource group
Container Apps environment name	cae-gemma-aca	ACA environment
Azure location	eastus	Must be a GPU-supported region
Container app name	gemma-api	Name of the deployed app
Workload profile name	NC8as-T4	GPU profile name
API username	admin	Username for Basic Auth
Created by	current shell user	Value for the created_by resource tag
API password	(required)	Password for Basic Auth — not echoed

The script provisions the following resources, all tagged with created_by:

1. Resource group
2. Log Analytics workspace
3. Container Apps environment
4. Container app (4 vCPU, 16 GB RAM, 1× T4 GPU)

At the end it prints your endpoint URL and credentials.

Cost Estimate

This project uses the NC8as-T4 GPU profile (1× NVIDIA T4, 16 GB VRAM) by default. All costs are for the eastus region with 1 replica running 24/7. The GPU profile is the dominant cost — the other resources are negligible by comparison.

NC8as-T4 GPU profile (default)

Resource	Per hour	Per day (24h)	Per month (730h)
Consumption-GPU-NC8as-T4	~$1.053	~$25.27	~$768
Log Analytics workspace	~$0.01	~$0.33	~$5–10
Container Apps environment	$0	$0	$0
Total estimate	~$1.06/hr	~$25.60/day	~$775–780/month

NC24-A100 GPU profile (higher performance)

Resource	Per hour	Per day (24h)	Per month (730h)
Consumption-GPU-NC24-A100	~$5.00	~$120.00	~$3,650

Reducing cost

Strategy	Saving
Stop the container when not in use	Up to 100% of GPU hours
Use gemma4:e2b instead of larger variants	No cost saving, but faster responses
Set --min-replicas 0	Scales to zero when idle — cold start on next request

Prices are approximate and subject to change. Check the Azure Pricing Calculator for current rates. GPU workload profiles are billed per second of active replica time.

Cold Start

The first time the container starts, it pulls the model from the Ollama registry. gemma4:e2b (~2 GB) takes about 1 minute; larger variants like gemma4:12b (~8 GB) take 2–3 minutes. Subsequent restarts are instant if the volume is retained.

Monitor pull progress:

az containerapp logs show --name gemma-api --resource-group rg-gemma-aca --follow

Sending Requests

All requests require HTTP Basic Auth (-u username:password).

OpenAI-compatible chat endpoint:

curl https://<FQDN>/v1/chat/completions \
  -u admin:yourpassword \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e2b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Ollama native generate:

curl https://<FQDN>/api/generate \
  -u admin:yourpassword \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e2b",
    "prompt": "Hello!",
    "stream": false
  }'

Streaming response:

curl https://<FQDN>/api/generate \
  -u admin:yourpassword \
  --no-buffer \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e2b",
    "prompt": "Tell me a story."
  }'

List loaded models:

curl https://<FQDN>/api/tags -u admin:yourpassword

Build Your Own Image

If you want to customise the image (different model, nginx config, etc.):

docker buildx build \
  --platform linux/amd64 \
  --provenance=false \
  -t <your-dockerhub-username>/gemma-ollama-auth:latest \
  --push .

--platform linux/amd64 is required — Azure's T4 nodes are x86_64. Building on Apple Silicon without this flag produces an arm64 image that won't run on the GPU node.

Cleanup

Run the destroy script to delete all provisioned resources:

chmod +x destroy.sh
./destroy.sh

The script will prompt for the resource group name and ask you to confirm by retyping it before deleting anything. This removes the resource group, container app, Container Apps environment, and Log Analytics workspace in one step.