A practical learning and engineering lab for Edge AI on the NVIDIA Jetson Orin Nano: real setup notes, verified commands, and applied chapters. Everything here either happened on the hardware or was checked against it.
| Chapter | Covers | Status |
|---|---|---|
setup/ |
Flashing, first access, Docker + the NVIDIA container runtime, headless mode, swap, Tailscale | Done |
docker-gpu/ |
A FastAPI service proving a real CUDA execution provider works inside a Jetson container: multi-stage Docker build, Compose health/restart behavior, native-vs-container benchmarks | Done |
Every chapter follows the same shape, described under "How this repo is organized" below.
Running AI models on edge devices is not only about model accuracy. A real Edge AI system also depends on hardware constraints, OS setup, GPU runtime configuration, containerized deployment, memory limits, inference performance, observability, and security. I document those details as I actually hit them, including the parts that went wrong, instead of writing them up as an idealized how-to.
- Device: Jetson Orin Nano (8GB)
- JetPack / L4T: 7.2 / r39.2. I chose the newest available stack over JetPack 6's production maturity because this is a learning repo. ADR-0001 has the full trade-off and the costs I accepted: less mature third-party tooling and thinner documentation.
- Docker + NVIDIA Container Toolkit: I set the NVIDIA runtime as Docker's default on this device (see
setup/), so GPU containers work here without extra flags. Chapters still show the--gpus/gpus:form explicitly, so the files work on a host without that default. - Base image:
docker-gpu's base image needs either--gpus allorNVIDIA_DISABLE_REQUIRE=1to get past a driver-compatibility check. ADR-0002 explains what each one actually does. - uv: Python dependency and project management, for both the root Jupyter environment and each chapter's own application code.
- Versions are chosen per chapter, not assumed to be shared. JetPack 7 is new enough that the guidance keeps shifting, so check a chapter's own ADRs before reusing anything from it.
I write up hard-to-reverse decisions as ADRs instead of burying them in a commit message or a code comment. Which JetPack version, which base image, how a project is laid out. Each one records the context that forced the decision, the options I considered and their downsides, the choice, and what it cost me. See docs/adr/README.md for the index and the process for adding one.
Every chapter has the same two core files, plus one shared command reference:
README.md— the front door: what it covers, why it matters for Edge AI, how to reproduce it, what can go wrong, what the trade-offs were.notes.md— the narrative log: what happened, what broke, what fixed it, in the order it actually happened. Not cleaned up into a tutorial after the fact.commands.md— a shared, repo-wide command reference.
Chapters add to that where the material needs it. docker-gpu/ also has fundamentals.md for the background theory, and a results/ directory holding the benchmark data and its write-up.
setup/scripts/verify-setup.sh is a read-only diagnostic script. Run it on the Jetson after setup to confirm the base environment is correct: Docker, the NVIDIA runtime, swap, and the L4T version.
These are my real setup notes and experiments. I hope they help someone going through the same path.