FBQueue (Forblaze Queue)

FBQueue is a lightweight, robust local job scheduler written in Rust. It is designed for researchers and developers who need efficient job management on shared computing resources without the complexity of system-wide installations.

💡 Why FBQueue?

FBQueue addresses the gap between manual script execution and heavy enterprise schedulers like Slurm or PBS.

• Polite Resource Sharing: Manage your own jobs "politely" on multi-user servers. Limit your personal resource consumption (CPU/GPU) to ensure fair access for everyone.
• Personal Scheduler in Restricted Environments: Get advanced scheduling (dependencies, priorities, walltime) on any server, even without root access or a system-wide scheduler.
• Enterprise-Grade Security & Simplicity:
  • No Network Ports: Operates entirely via the file system. No firewall rules or port exposures required.
  • No Database: Uses a transparent file-based state management system.
  • Auto-Shutdown Daemon: The daemon automatically terminates after a configurable period of inactivity (default 300s). It only shuts down when both the running and pending queues are completely empty.
• Zero-Config & Zero Management: A single, dependency-free binary that runs in user-space. The daemon starts automatically on any command (sub, stat, del) and stops when idle—no manual daemon management required.

🎨 Key Features

• Resource-Aware Scheduling: Flexible control using abstract capacity and job cost.
• Multi-Queue & Priority Support: Configure multiple queues with different priorities and limits.
• PBS/HPC Compatibility: Supports PBS-style commands and parses embedded script directives (#PBS, #SBATCH, etc.). Detailed usage is available in the MANUAL.
• Batch Processing: Simplified submission for parameter studies using --range and --list.
• Resilience: Automatically recovers and resumes interrupted jobs after a system reboot or daemon restart.

🚀 Quick Start

Installation

Linux

Download the pre-built static binary and place it in your $PATH:

# 1. Download
wget https://github.com/ForblazeProject/fbqueue/releases/download/v0.9.3/fbqueue-linux-x64.tar.gz

# 2. Extract and move
tar -xzvf fbqueue-linux-x64.tar.gz
mkdir -p ~/bin
mv fbqueue/fbqueue ~/bin/

# 3. (Optional) Create PBS-style symbolic links
ln -s ~/bin/fbqueue ~/bin/qsub
ln -s ~/bin/fbqueue ~/bin/qstat
ln -s ~/bin/fbqueue ~/bin/qdel

Windows

1. Download fbqueue-windows-x64.zip from Releases.
2. Extract the ZIP and move fbqueue.exe to a folder in your Path.

Job Submission (sub)

FBQueue automatically handles path prefixes and shell selection.

# Linux
fbqueue sub my_script.sh

# Windows
fbqueue sub my_script.ps1

Status & Management

fbqueue stat          # Check active jobs and resource usage
fbqueue stat -H       # Show recent job history (last 100 jobs)
fbqueue del <job_id>  # Delete/Cancel a job
fbqueue sub -a +1h ./task.sh  # Schedule a job to start in 1 hour

📂 Environment & Project Isolation

By default, FBQueue stores its data in ~/.fbqueue/ (the user's home directory), keeping your queue private.

For team collaboration on a single machine, you can point multiple users to a local shared directory using the FBQUEUE_DIR environment variable. Note that using network-mounted drives (NFS/SMB) is discouraged due to potential file-locking latency:

# Example: Shared local project directory
export FBQUEUE_DIR=/var/lib/fbqueue/project_a
fbqueue sub ./calc.sh

📜 Change Log

v0.9.3

• Full PBS Environment Emulation: Jobs now automatically define standard PBS variables like $PBS_JOBID, $PBS_O_WORKDIR, $PBS_O_HOST, and $PBS_O_LOGNAME.
• Nodefile Support: Dynamically generates and cleans up $PBS_NODEFILE for compatibility with scripts expecting a list of execution nodes.
• Improved Metadata Accuracy: Enhanced hostname and user detection across both Linux and Windows for more reliable job logging.

v0.9.2

• Fixed Process Leak: Improved job cancellation logic using process groups (setsid) on Unix and tree termination (/T) on Windows. Subprocesses spawned by job scripts are now correctly terminated.
• Robust Job Filtering: qstat <jobID> (and stat <jobID>) now works reliably even if the job has already finished and moved to history. It also correctly handles the .master suffix.
• Dependency Update: Added libc dependency for Unix systems to support advanced process management.

v0.9.1

• Enhanced PBS Compatibility: Added support for qstat -u <user> filtering.
• Improved History Display: qstat -H now correctly shows job history in PBS-style format.
• PBS-Style Submission: qsub now outputs only the job ID (e.g., 1.master) upon success, matching traditional HPC behavior.

v0.9.0

• International Release: Documentation translated to English and terminology standardized.
• Delayed Start: Added human-readable time support for -a option (e.g., +1h, 18:00).
• Resource Efficiency: Added configurable inactivity_timeout and automatic background archiving of old jobs.
• Zero Management: Daemon starts automatically on any command and shuts down when idle.

---

Author

Forblaze Project
Website: https://forblaze-works.com/en/

License

This project is licensed under the MIT License - see the LICENSE file for details.