pvmlab: your local pxeboot virtual lab

This project provides a command-line tool, pvmlab, to automate the setup of a simple virtual provisioning lab on macOS. It uses QEMU, socket_vmnet, cloud-init, and Docker to create and manage an environment where multiple "target" VMs are deployed in a private virtual network and one VM functions as the "provisioner" VM offering pxeboot services and working as default gw for internet access for those VMs.

All generated artifacts (VM disks, ISOs, logs, etc.) are stored neatly in ~/.pvmlab/, keeping the project repository clean.

Use Cases

This virtual lab is ideal for a variety of network-based provisioning and testing scenarios, such as:

• Testing OS Installers: Develop and test automated OS installation configurations like Ubuntu Autoinstall, Debian Preseed, or Red Hat Kickstart. Or test your own ;)
• Developing Network Boot Environments: Experiment with and develop iPXE scripts or other network boot setups.
• Simulating Bare-Metal Provisioning: Mimic the process of provisioning physical servers in a fully virtualized, local environment.
• Network Service Testing: Safely test DHCP, TFTP, and HTTP services in an isolated network.
• Playing with Kubernetes or Other Orchestration Tools: Use the lab to experiment with cluster provisioning and management tools like kubernetes, Tinkerbell, Canonical MAAS, etc.
• Learning and Experimentation: Provide a hands-on environment for learning about PXE boot, network automation, and cloud-init.

Core Concepts

Features

• Automated VM Creation: Quickly create Ubuntu VMs from cloud images.
• Provisioner/Target Architecture: Set up a dedicated "provisioner" VM to serve network resources (DHCP, PXE boot) to multiple "target" VMs.
• Private Networking: Uses socket_vmnet to create an isolated virtual network for your lab environment.
• Dual-stack Networking: Configure VMs with both IPv4 and IPv6 addresses on the private network.
• Direct SSH Access: Connect directly to any VM (provisioner or target) with a single command.
• Simple CLI: Manage the entire lab lifecycle with intuitive pvmlab commands.

Architecture

For a detailed explanation of the project's architecture, features, and VM roles, please see the Architecture documentation.

Getting Started

Installation

To install pvmlab on macOS, the recommended method is using Homebrew:

1. Install with Homebrew:

   brew tap pallotron/pvmlab
   brew install pvmlab
   sudo pvmlab system setup-launchd

   This will install pvmlab and all its dependencies (QEMU, cdrtools, socat, socket_vmnet, go, docker).

Manual Installation (Alternative)

If you prefer to install manually, or are on a different operating system, follow these steps:

1. Install Dependencies: You must install the following dependencies manually. On macOS, you can use Homebrew for this:

   brew install qemu cdrtools socat socket_vmnet docker go

   Note: docker refers to the Docker CLI, which is included with Docker Desktop for Mac.

2. Clone the Repository:

   git clone https://github.com/pallotron/pvmlab.git
   cd pvmlab

3. Build and Install: This command compiles the pvmlab CLI, builds the pxeboot stack Docker container, and sets up shell completion.

   make install

4. Set up the Lab Environment: This command creates the ~/.pvmlab/ directory structure and generates an SSH key for accessing the VMs. It may require your sudo password to configure socket_vmnet.

   pvmlab setup

5. Source Shell Completions (Optional but Recommended): The make all command attempts to install shell completions. If it doesn't work for your shell, you may need to source them manually. Either open a new terminal or reload your shell's rc file.

   • For Zsh: source ~/.zshrc
   • For Bash: source ~/.bashrc

After installation, you can run the command as pvmlab from any directory.

Usage: A Typical Workflow

Here is a step-by-step guide to a common workflow.

Step 1: Start the Network Service

This service manages the private virtual network. You may be prompted for your password. The make install command should have already set this up for you, but run this commands to ensure.

pvmlab socket_vmnet status
# if not running, start it:
pvmlab socket_vmnet start

Step 2: Create the VMs

This command downloads cloud images, creates VM disks, and generates cloud-init configurations.

First, create the provisioner VM, which will serve network resources (DHCP, PXE boot) and act as the default gateway for other VMs:

pvmlab provisioner create provisioner --ip 192.168.100.1/24 --ipv6 fd00:cafe:babe::1/64

Then, create your target VMs:

pvmlab vm create client1
pvmlab vm create client2 --arch x86_64 --pxeboot

You can create x86_64 or aarch64 VMs by specifying the --arch flag. By default, aarch64 is used. You can also use --disk (default) or --pxeboot flags to customize how the VM should boot.

Step 3: Manage the VMs

Once created, you can start, stop, and interact with your VMs.

Start VMs:

pvmlab vm start provisioner
pvmlab vm start client1

The start command accepts --interactive mode, which attaches your terminal to the VM's console. You can provide the --wait flag to block the terminal until the VM has reached cloud-init.target. Providing no flags starts the VM in the background. You can monitor logs via the pvmlab vm logs command (see below).

List VMs:

❯ pvmlab vm list
┌─────────────┬─────────┬───────────┬───────────────┬───────────────────┬───────────────────┬─────────┐
│    NAME     │  ARCH   │ BOOT TYPE │  PRIVATE IP   │   PRIVATE IPV 6   │        MAC        │ STATUS  │
├─────────────┼─────────┼───────────┼───────────────┼───────────────────┼───────────────────┼─────────┤
│ client1     │ x86_64  │ disk      │ 192.168.100.2 │ fd00:cafe:babe::2 │ 0a:5d:ea:18:55:b6 │ Running │
│ provisioner │ aarch64 │ disk      │ 192.168.100.1 │ fd00:cafe:babe::1 │ 62:a8:6a:f6:5b:3c │ Running │
└─────────────┴─────────┴───────────┴───────────────┴───────────────────┴───────────────────┴─────────┘

Access a VM's Shell:

# Access the Provisioner VM
pvmlab vm shell provisioner

# Access a Target VM
pvmlab vm shell client1

Monitor Logs:

pvmlab vm logs provisioner
pvmlab vm logs client1

Stop VMs:

pvmlab vm stop provisioner
pvmlab vm stop client1

Step 4: Clean Up

You can clean up individual VMs or the entire lab environment.

Clean a single VM: This stops the VM and removes all its associated files from ~/.pvmlab/.

pvmlab vm clean provisioner

Clean the entire lab: This removes everything, including the socket_vmnet service and the ~/.pvmlab/ directory.

pvmlab clean

Artifacts Directory

All files generated by pvmlab are stored in a hidden directory in your home folder to keep the project's working directory clean.

The structure of this directory is as follows:

~/.pvmlab/
├── configs/        # Generated cloud-init ISO files (.iso) for each VM
├── docker_images/  # Docker images saved as .tar files to be shared with the provisioner VM
├── images/         # Downloaded cloud image templates, pxeboot assets, and rootfs images for pxeboot
├── logs/           # VM console logs
├── monitors/       # QEMU monitor sockets for interacting with the hypervisor
├── pids/           # Process ID files for running VMs
├── ssh/            # Generated SSH key pair (vm_rsa, vm_rsa.pub) for VM access
└── vms/            # VM disk images (.qcow2) created from the base images

Missing Features

Missing features are being tracked as issues in the GitHub repository. Please feel free to contribute!

Advanced Topics & Further Reading

For more detailed information, please refer to the documentation in the docs/ directory:

• CLI Command Reference: A full list of all commands and their flags.
• Architecture: A detailed explanation of the project's design and components.
• Interacting with the pxeboot Container: How to manage the pxeboot Docker container inside the provisioner VM.