Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.local.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ NETRA_PUBLIC_URL=__PUBLIC_URL__
# for robots on the same network (NETRA_LAN_BIND=127.0.0.1 closes it).
# NETRA_TUNNEL_PORT=8088
# NETRA_LAN_BIND=0.0.0.0
# Next to other stacks on one server (Mac mini with Taksu, Molly, monitoring): NETRA's own block,
# loopback only, robots through the tunnel. scripts/local.sh ports shows what is free.
# NETRA_TUNNEL_PORT=4490
# NETRA_LAN_PORT=4491
# NETRA_LAN_BIND=127.0.0.1

POSTGRES_PASSWORD=__POSTGRES_PASSWORD__
GRAFANA_PASSWORD=__GRAFANA_PASSWORD__
Expand Down
44 changes: 43 additions & 1 deletion LOCAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,46 @@ closes it).
Third-party apps use the integration API through the same hostname: docs at
`https://netra.example.com/api/ext/docs`, clients in **Settings → API clients** (README).

## Next to other stacks

On a computer that already runs other services (the Mac mini with Taksu, the Molly dashboard and
the server monitoring), NETRA must not take their ports. Only Caddy publishes ports; the database,
API, Grafana, Prometheus and NATS stay inside Docker, and the compose project `netra-fleet` keeps
its own containers, network and volumes. Give NETRA its own block on the loopback and let robots
and people come in through the tunnel:

```bash
# .env
COMPOSE_FILE=docker-compose.yml:docker-compose.local.yml:docker-compose.tunnel.yml
NETRA_PUBLIC_URL=https://netra.example.com
NETRA_TUNNEL_PORT=4490
NETRA_LAN_PORT=4491
NETRA_LAN_BIND=127.0.0.1
```

| Port | Used by | |
|---|---|---|
| 80, 8080 | other services on the Mac mini | never for NETRA there |
| 3000, 3100, 9090 | server Grafana, Loki, Prometheus | NETRA's own Grafana and Prometheus are not published |
| 4470 | Molly dashboard | `127.0.0.1` |
| 4480 | Taksu (Caddy) | `127.0.0.1` |
| 4490, 4491 | NETRA tunnel and loopback LAN port | `127.0.0.1` |
| 11434 | Ollama, native, shared | NETRA's vision model and the other stacks' models share its memory |

`scripts/local.sh up` and `start` stop with the name of the program or compose project holding a
port before anything starts; `scripts/local.sh ports` and `doctor` show the same. In
`~/.cloudflared/config.yml` each hostname points at its stack's port:

```yaml
ingress:
- hostname: netra.example.com
service: http://127.0.0.1:4490
- service: http_status:404
```

On a laptop at a site, keep the defaults (port 80 on every interface) so robots on the same Wi-Fi
reach NETRA directly.

## Everyday commands

| Command | |
Expand All @@ -133,6 +173,7 @@ Third-party apps use the integration API through the same hostname: docs at
| `scripts/local.sh start` | start the images built before, without internet (demo day) |
| `scripts/local.sh status` | containers and API health |
| `scripts/local.sh lan` | the Mac's address on every network, for `CC_URL` and phones |
| `scripts/local.sh ports` | ports NETRA publishes and whether another program holds them |
| `scripts/local.sh logs [api\|ai\|web\|db\|...]` | follow logs |
| `scripts/local.sh doctor` | check Docker, Ollama, model, AI link, LAN reachability, firewall, sleep |
| `scripts/local.sh export [file]` | database, photos and floor plans in one file |
Expand All @@ -154,6 +195,7 @@ daily database dumps go to `backups/`, exports to `exports/`.
| `NETRA_LANG` | language of Telegram captions (`en`, `id`) |
| `NETRA_LAN_PORT` | port for the dashboard and robots (80); add it to `NETRA_PUBLIC_URL` when changed |
| `NETRA_TUNNEL_PORT`, `NETRA_LAN_BIND` | tunnel only: local port for cloudflared (8088), LAN side (`0.0.0.0`) |
| (shared server) | `NETRA_TUNNEL_PORT=4490`, `NETRA_LAN_PORT=4491`, `NETRA_LAN_BIND=127.0.0.1`, see [Next to other stacks](#next-to-other-stacks) |
| `NETRA_EXT_TOKEN_MINUTES` | integration API access token lifetime (60) |

After editing `.env`: `scripts/local.sh up`.
Expand All @@ -164,7 +206,7 @@ After editing `.env`: `scripts/local.sh up`.
|---|---|
| `doctor`: AI container cannot reach Ollama | `brew services stop ollama && OLLAMA_HOST=0.0.0.0 ollama serve` |
| Robot stays offline | on the robot `curl http://<address>/healthz`; no answer: other network, venue Wi-Fi isolating devices, or the macOS firewall (allow Docker); `401` in `logs api`: wrong `CC_TOKEN` |
| Port 80 already in use | stop the other web server (`sudo lsof -i :80`) or set `NETRA_LAN_PORT` |
| `port … is used by …` at `up` | another stack or program has the port: set `NETRA_LAN_PORT` / `NETRA_TUNNEL_PORT` ([Next to other stacks](#next-to-other-stacks)) |
| `git pull` refuses: untracked `docker-compose.tunnel.yml` / `Caddyfile.tunnel` | delete the two hand-made files, the repository has them now |
| Anomalies stay "Analyzing" for long | the model is loading or too large for the RAM: use `qwen2.5vl:3b` |
| Schedules did not run | the Mac slept: `scripts/local.sh awake`, keep the lid open or use an external display |
Expand Down
91 changes: 90 additions & 1 deletion scripts/local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
# scripts/local.sh lan addresses robots and phones can use on each network
# scripts/local.sh export [file] all data (database, photos, floor plans) into one file
# scripts/local.sh import <file> replace the data here with an export (demo laptop)
# scripts/local.sh ports ports NETRA publishes on this computer, and who holds them
# scripts/local.sh status | logs [service] | doctor | awake | down | reset
#
# Works with the bash 3.2 that ships with macOS.
Expand Down Expand Up @@ -63,6 +64,79 @@ lan_port() {
port_suffix() { [[ $(lan_port) == 80 ]] || echo ":$(lan_port)"; }
BASE_URL="http://localhost$(port_suffix)"

# Tunnel port (NETRA_TUNNEL_PORT, default 8088), only published with docker-compose.tunnel.yml.
tunnel_on() { [[ $(envget COMPOSE_FILE) == *docker-compose.tunnel.yml* ]]; }
tunnel_port() {
local port
port=$(envget NETRA_TUNNEL_PORT)
echo "${port:-8088}"
}

# Host ports NETRA publishes ("name port"); every other service stays inside Docker.
netra_ports() {
echo "lan $(lan_port)"
if tunnel_on; then echo "tunnel $(tunnel_port)"; fi
}

# Something accepts connections on the port (works for listeners of any user, unlike lsof).
port_busy() { (exec 3<>"/dev/tcp/127.0.0.1/$1") 2>/dev/null; }

# The port is published by this stack's own containers (a restart keeps it).
port_ours() {
[[ -n $(docker ps -q --filter label=com.docker.compose.project=netra-fleet --filter "publish=$1" 2>/dev/null) ]]
}

# Who holds a port: the compose project or process name when visible, else a hint.
port_holder() {
local project name
project=$(docker ps --filter "publish=$1" --format '{{.Label "com.docker.compose.project"}} ({{.Names}})' 2>/dev/null | head -1)
if [[ -n $project ]]; then
echo "Docker: $project"
return
fi
name=$(lsof -nP -iTCP:"$1" -sTCP:LISTEN -Fc 2>/dev/null | sed -n 's/^c//p' | head -1)
echo "${name:-another program (sudo lsof -nP -iTCP:$1 -sTCP:LISTEN)}"
}

# Stop before compose fails half way when another stack or program already uses a NETRA port.
check_ports() {
local name port clash=''
while read -r name port; do
if port_busy "$port" && ! port_ours "$port"; then
bad "port $port ($name) is used by $(port_holder "$port")"
clash=1
fi
done < <(netra_ports)
[[ -z $clash ]] && return 0
printf ' Pick free ports in .env (NETRA_LAN_PORT, NETRA_TUNNEL_PORT), see LOCAL.md
' >&2
printf ' "Next to other stacks", then run this again.
' >&2
exit 1
}

cmd_ports() {
need_env
local name port state bind
bind=$(envget NETRA_LAN_BIND)
title "Ports NETRA publishes on this computer"
while read -r name port; do
if port_ours "$port"; then
state="${G}NETRA${N}"
elif port_busy "$port"; then
state="${R}used by $(port_holder "$port")${N}"
else
state="free"
fi
case $name in
lan) printf ' %-6s %-24s %s\n' "$port" "dashboard and robots (${bind:-0.0.0.0})" "$state" ;;
tunnel) printf ' %-6s %-24s %s\n' "$port" "cloudflared (127.0.0.1)" "$state" ;;
esac
done < <(netra_ports)
printf ' %sEverything else (database, API, Grafana, Prometheus, NATS) stays inside Docker.%s\n' "$D" "$N"
printf ' %sOllama on 11434 is shared with other stacks on this computer.%s\n' "$D" "$N"
}

# "interface address" for every IPv4 address a robot could reach (no loopback or VPN tunnels).
lan_addrs() {
ifconfig 2>/dev/null | awk '
Expand Down Expand Up @@ -186,6 +260,7 @@ cmd_setup() {
cmd_up() {
need_env
docker info >/dev/null 2>&1 || die "Docker Desktop is not running: open -a Docker"
check_ports
start_ollama || warn "Ollama is not running: final detection falls back to the robot result"
title "Starting NETRA Fleet (first build takes a few minutes)"
docker compose up -d --build --remove-orphans
Expand All @@ -197,6 +272,7 @@ cmd_up() {
cmd_start() {
need_env
docker info >/dev/null 2>&1 || die "Docker Desktop is not running: open -a Docker"
check_ports
start_ollama || warn "Ollama is not running: final detection falls back to the robot result"
title "Starting NETRA Fleet (no rebuild)"
docker compose up -d --no-build --remove-orphans || die "images missing: run scripts/local.sh up once with internet"
Expand Down Expand Up @@ -435,6 +511,18 @@ cmd_doctor() {
bad "AI container cannot reach Ollama: brew services stop ollama && OLLAMA_HOST=0.0.0.0 ollama serve"
fi

title "Ports"
local name port
while read -r name port; do
if port_ours "$port"; then
ok "$name port $port: NETRA"
elif port_busy "$port"; then
bad "$name port $port used by $(port_holder "$port") (scripts/local.sh ports)"
else
ok "$name port $port free"
fi
done < <(netra_ports)

title "Stack"
if curl -fsS --max-time 3 "$BASE_URL/healthz" >/dev/null 2>&1; then
ok "API healthy on $BASE_URL"
Expand Down Expand Up @@ -504,10 +592,11 @@ status) cmd_status ;;
logs) shift && cmd_logs "$@" ;;
robot) shift && cmd_robot "$@" ;;
lan) cmd_lan ;;
ports) cmd_ports ;;
export) shift && cmd_export "$@" ;;
import) shift && cmd_import "$@" ;;
doctor) cmd_doctor ;;
awake) cmd_awake ;;
reset) cmd_reset ;;
*) sed -n '2,11p' scripts/local.sh | sed 's/^# \{0,1\}//' ;;
*) sed -n '2,12p' scripts/local.sh | sed 's/^# \{0,1\}//' ;;
esac
Loading