Skip to content
elgatopanzonPublic

About

Automate Vast.ai instances with IaC

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

127 Commits

Folders and files

Repository files navigation

vastup

vastup is a Bash CLI for finding, creating, managing, and cleaning up Vast.ai GPU instances. It wraps the vastai CLI with opinionated defaults for repeatable project runs: YAML configs, offer filtering, SSH setup, file transfer, lifecycle hooks, log following, tunnels, and remote watchdog cleanup.

The goal is to make short-lived GPU jobs reproducible without turning every Vast.ai session into a pile of one-off shell commands.

Status

vastup is early but usable. It is designed for people who already understand Vast.ai costs and instance lifecycle behavior. Always review the selected offer and watchdog settings before running expensive jobs.

Requirements

  • Bash for vastup.
  • POSIX sh for the remote watchdog script.
  • vastai, installed manually or with vastup --install-vastai.
  • jq, curl, awk, base64, cksum, timeout, tput, and common core utilities.
  • One YAML parser for vastup.yml/vastup.yaml: yq, Ruby with YAML, or Python with PyYAML.
  • A Vast.ai API key in VAST_API_KEY for account operations. Remote watchdog actions also read ~/.vast_api_key on the instance when they need to stop or destroy the instance.

Install

Clone the repository and put vastup on your PATH:

git clone https://github.com/elgatopanzon/vastup.git
cd vastup
chmod +x vastup scripts/vastup-watchdog.sh

Run it from the repo:

./vastup --help

Or symlink it somewhere on PATH:

ln -s "$PWD/vastup" "$HOME/.local/bin/vastup"

Mental Model

vastup has two main modes.

Direct mode:

vastup search --gpu 4090 --vram 24 --max-price 1.00
vastup run pytorch/pytorch --gpu 4090 --vram 24 --pick-best --no-confirm

Project mode:

cp examples/pytorch-notebook.yml vastup.yml
vastup check
vastup up
vastup logs -f
vastup down

In project mode, vastup looks for vastup.yml or vastup.yaml. The YAML is converted into the same arguments used by vastup run. The instance name is derived from the configured name or image plus a hash of the local machine and project path, so different projects do not accidentally target the same instance.

Common Commands

Search offers:

vastup search --gpu 4090 --gpu-count 1 --vram 24 --cuda 12.1 --max-price 1.00

Create an instance directly:

vastup run pytorch/pytorch --ssh --gpu 4090 --vram 24 --disk 80

Validate project configuration:

vastup check
vastup check --json

Create or resume the project instance:

vastup up

Recreate when YAML or image config changed:

vastup up --auto-recreate

Follow logs:

vastup logs -f
vastup logs --daemon

Attach to the managed tmux session:

vastup attach

Transfer files:

vastup push INSTANCE_OR_NAME ./data /workspace/data
vastup pull INSTANCE_OR_NAME /workspace/out ./out

Open an SSH tunnel:

vastup tunnel INSTANCE_OR_NAME 8080:7860

Destroy the project instance:

vastup down

Preview destructive cleanup:

vastup prune --all --min-cost 1.00 --inactive-for 2h --dryrun

YAML Project Files

A project file can put run configuration at the top level:

name: my-gpu-job
image: pytorch/pytorch:latest
resources:
  gpu:
    models:
      - 4090
    vram: 24
    count: 1
  disk:
    size: 80
offer:
  max_price: 1.00
  reliability: 95
  safe: true
destroy_failed: true
watchdog:
  action: destroy
  max_lifetime: 4h

Or inside a top-level run object:

run:
  name: my-gpu-job
  image: pytorch/pytorch:latest
  offer:
    pick_best: true
    no_confirm: true

YAML files support extends, relative local paths for hooks and transfers, and Docker Compose-style environment interpolation such as ${VAR:-default}.

Examples

The examples/ directory contains focused project files:

  • examples/nginx.yml: minimal HTTP service with a direct port and watchdog.
  • examples/pytorch-notebook.yml: GPU notebook-style instance with a tunnel and idle watchdog.
  • examples/batch-job.yml: detached batch command with file push/pull and destroy-on-exit.
  • examples/hook-workflow.yml: local and remote lifecycle hooks.

Copy one to vastup.yml, edit the image/resources, then run:

vastup check
vastup up

Hooks And Bootstrap

vastup can run local hooks before and after instance creation, and remote hooks around bootstrap, start, and exit. Use hooks for setup that belongs to the run lifecycle: generating config, preparing datasets, notifying another system, or collecting results.

Bootstrap commands and scripts run after SSH is ready and before the managed entrypoint starts.

Watchdog

The remote watchdog can stop or destroy an instance after a lifetime, spend budget, hard max lifetime, or idle window. Activity can be detected from SSH sessions, process names, GPU usage, CPU usage, network rate, disk rate, file mtimes, or a custom activity hook.

Prefer --watchdog-action destroy for disposable jobs. Use stop only when you intend to keep the instance disk and understand the ongoing storage cost.

Cost And Safety Notes

  • Vast.ai pricing varies by host and offer. Review price, storage, bandwidth, reliability, and duration before creating an instance.
  • Stopped instances may still incur storage cost. Destroy disposable instances when done.
  • prune, down, rm, watchdog actions, and --rm can destroy instances. Use --dryrun when checking behavior.
  • --pick-best --no-confirm is convenient for automation, but it removes the final human confirmation.

Development Checks

For local changes:

bash -n vastup
sh -n scripts/vastup-watchdog.sh
./vastup --help

If a YAML parser is installed:

./vastup check --project-file vastup.example.yml --json

License

MIT. See LICENSE.

About

Automate Vast.ai instances with IaC

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages