geeup: Simple CLI for Earth Engine Uploads

geeup is a comprehensive command-line tool designed to simplify data preparation and batch uploading to Google Earth Engine. Handle images, shapefiles, and metadata with retry logic, parallel processing, and enhanced path normalization.

Citation

If you use this project, please star and cite it as below:

Samapriya Roy. (2025). samapriya/geeup: geeup: Simple CLI for Earth Engine Uploads (2.0.0).
Zenodo. https://doi.org/10.5281/zenodo.18073520

Table of Contents

• geeup: Simple CLI for Earth Engine Uploads
  • Citation
  • Table of Contents
  • Installation
    • Method 1: PyPI (Recommended)
    • Method 2: From Source
  • Getting Started
  • Authentication
    • Cookie Setup
    • Service Account Configuration
  • Data Preparation
    • Rename Files
    • Zip Shapefiles
    • Generate Metadata
  • Batch Upload
    • Upload Images
    • Upload Tables (Tabup)
  • Monitoring
    • Check Quota
    • Task Status
    • Cancel Tasks
  • Utilities
    • Delete Assets
    • Open Documentation
  • Full Documentation
  • Changelog
    • Version 2.0.0
  • Contributing
  • License
  • Acknowledgments

Installation

Method 1: PyPI (Recommended)

pip install geeup

Method 2: From Source

git clone https://github.com/samapriya/geeup.git
cd geeup
pip install -e .

Requirements:

• Python 3.7+
• Earth Engine Python API
• Authenticated Earth Engine account (earthengine authenticate)

Getting Started

View all available commands:

geeup -h

Main interface output:

usage: geeup [-h] {readme,quota,auth,rename,zipshape,getmeta,cookie_setup,upload,tabup,tasks,cancel,delete} ...

Simple Client for Earth Engine Uploads with Enhanced UI and Service Account Support

positional arguments:
  {readme,quota,auth,rename,zipshape,getmeta,cookie_setup,upload,tabup,tasks,cancel,delete}
    readme              Open the geeup documentation page
    quota               Print Earth Engine storage and asset count quota with visual progress bars
    auth                Configure service account authentication
    rename              Rename files to adhere to EE naming rules with interactive preview
    zipshape            Zip all shapefiles and subsidiary files in folder with progress tracking
    getmeta             Create generalized metadata for rasters in folder with progress tracking
    cookie_setup        Setup cookies to be used for upload
    upload              Batch Image Uploader for uploading TIF files to a GEE collection
    tabup               Batch Table Uploader for uploading shapefiles/CSVs to a GEE folder
    tasks               Query current task status with rich formatting [completed, running, ready, failed, cancelled]
    cancel              Cancel all, running, pending tasks or specific task ID with progress tracking
    delete              Delete collection and all items inside recursively

Authentication

Cookie Setup

The cookie_setup command configures authentication cookies required for uploads. This uses the Copy Cookies Chrome extension.

Setup Steps:

1. Install the Copy Cookies extension in Chrome
2. Log into code.earthengine.google.com
3. Click the extension icon to copy cookies
4. Run the setup command and paste when prompted:

geeup cookie_setup

Important Notes:

• Use a fresh browser window (not incognito)
• Clear cookies and retry if you get "Unable to read" errors
• Ensure cookies match the account initialized with earthengine authenticate

Platform-Specific Instructions:

For Bash users experiencing paste limitations:

stty -icanon
geeup cookie_setup
stty icanon

For Mac users, you may need to change your default shell from /bin/zsh to /bin/sh.

Service Account Configuration

The auth command manages service account authentication for automated workflows.

Initialize with service account:

geeup auth --cred "/path/to/service-account.json"

Check authentication status:

geeup auth --status

Remove credentials:

geeup auth --remove

Arguments:

• --cred: Path to service account JSON credentials file (optional)
• --status: Show current authentication status (optional)
• --remove: Remove stored service account credentials (optional)

Data Preparation

Rename Files

Sanitize filenames to adhere to Earth Engine naming conventions. Removes spaces and special characters.

geeup rename --input "./raw_data" --batch

Arguments:

• --input: Path to directory with files to rename (required)
• --batch: Skip confirmation and rename all files automatically (optional)

Interactive Mode: By default, shows a preview of changes and asks for confirmation before renaming.

Zip Shapefiles

Package shapefile components (.shp, .shx, .dbf, .prj) into individual ZIP archives for upload.

geeup zipshape --input "./vectors" --output "./zipped_vectors"

Arguments:

• --input: Path to directory containing raw shapefiles (required)
• --output: Destination folder for ZIP archives (required)

Generate Metadata

Create a metadata CSV file for rasters, defining properties like image IDs, dimensions, and data types.

geeup getmeta --input "./rasters" --metadata "./metadata.csv"

Arguments:

• --input: Path to directory containing raster (TIF) files (required)
• --metadata: Full path for exported metadata CSV file (required)

Generated Fields:

• system:index: Asset ID (filename without extension)
• xsize, ysize: Raster dimensions
• num_bands: Number of bands
• data_type: Data type (e.g., Byte, Float32)
• color_interpretation: Color interpretation (if not "Undefined")
• inferred_kind: Inferred semantic type (image, categorical, continuous)

Batch Upload

Upload Images

Batch upload GeoTIFF files to an Earth Engine Image Collection with advanced features like retry logic and parallel processing.

Basic Usage:

geeup upload \
  --source "./rasters" \
  --dest "users/username/collection" \
  --metadata "./metadata.csv" \
  --user "email@gmail.com"

Advanced Usage with Parallel Workers:

geeup upload \
  --source "./rasters" \
  --dest "users/username/collection" \
  --metadata "./metadata.csv" \
  --user "email@gmail.com" \
  --workers 4 \
  --resume \
  --pyramids MEAN \
  --nodata -9999

Arguments:

Argument	Type	Description
--source	Required	Path to directory with images for upload
--dest	Required	Destination path for GEE Image Collection (e.g., users/you/collection)
--metadata, -m	Required	Path to metadata CSV file
--user, -u	Required	Google account email address
--nodata	Optional	Integer value to burn as NoData (missing data)
--mask	Optional	Boolean (True/False). Use last band as validity mask
--pyramids	Optional	Pyramiding policy: MEAN, MODE, MIN, MAX, SAMPLE (default: MEAN)
--overwrite	Optional	Overwrite existing assets (pass 'yes' or 'y')
--dry-run	Optional	Run validation without uploading
--workers	Optional	Number of parallel upload workers (default: 1)
--max-inflight-tasks	Optional	Maximum concurrent EE tasks (default: 2800)
--resume	Optional	Resume from previous state
--retry-failed	Optional	Retry only failed uploads

Features:

• Path Normalization: Automatic handling of legacy (users/) and cloud project paths
• Collection Auto-Creation: Prompts to create missing collections and parent folders
• State Persistence: Saves progress to .geeup-state.json for resumption
• Parallel Uploads: Multi-threaded GCS uploads with progress bars
• Task Throttling: Automatically waits when task limit is reached

Upload Tables (Tabup)

Batch upload table data (vectors) from zipped shapefiles or CSVs to a GEE folder.

Basic Usage:

geeup tabup \
  --source "./zipped_vectors" \
  --dest "users/username/folder" \
  --user "email@gmail.com"

CSV with Geometry Columns:

geeup tabup \
  --source "./csv_files" \
  --dest "users/username/folder" \
  --user "email@gmail.com" \
  --x "longitude" \
  --y "latitude"

Arguments:

Argument	Type	Description
--source	Required	Path to directory with zipped files or CSV files
--dest	Required	Destination path for GEE Folder (e.g., users/you/folder)
--user, -u	Required	Google account email address
--x	Optional	Column name containing longitude values (for CSVs)
--y	Optional	Column name containing latitude values (for CSVs)
--metadata	Optional	Path to CSV with metadata
--overwrite	Optional	Overwrite existing assets (pass 'yes' or 'y')
--dry-run	Optional	Run validation without uploading
--workers	Optional	Number of parallel workers (default: 1)
--max-inflight-tasks	Optional	Maximum concurrent EE tasks (default: 2500)
--resume	Optional	Resume from previous state
--retry-failed	Optional	Retry only failed uploads
--max-error-meters	Optional	Maximum geometry error in meters (default: 1.0)
--max-vertices	Optional	Maximum vertices per geometry (default: 1000000)

Features:

• Supports both zipped shapefiles and CSV files
• Automatic folder creation with confirmation
• State persistence for resumable uploads
• Progress tracking with file-level details

Monitoring

Check Quota

Display Earth Engine storage usage and asset counts with visual progress bars.

Check all quotas:

geeup quota

Check specific project:

geeup quota --project "projects/my-project"

Arguments:

• --project: Project name in format projects/project-name/assets/ (optional)

Output includes:

• Storage usage (bytes/GB)
• Asset count
• Visual progress bars
• Support for both legacy and cloud projects

Task Status

Query the status of Earth Engine tasks with rich formatting.

Summary of all tasks:

geeup tasks

Filter by state:

geeup tasks --state RUNNING

Query specific task:

geeup tasks --id "TASK_ID_HERE"

Arguments:

• --state: Filter by state (COMPLETED, READY, RUNNING, FAILED, CANCELLED)
• --id: Query specific task ID

Cancel Tasks

Cancel Earth Engine tasks with progress tracking.

Cancel all tasks:

geeup cancel --tasks all

Cancel running tasks:

geeup cancel --tasks running

Cancel pending tasks:

geeup cancel --tasks pending

Cancel specific task:

geeup cancel --tasks "TASK_ID_HERE"

Arguments:

• --tasks: Specify 'all', 'running', 'pending', or a specific task ID (required)

Utilities

Delete Assets

Recursively delete an Earth Engine asset (collection, folder, or image).

geeup delete --id "users/username/test_collection"

Arguments:

• --id: Full path to asset for deletion (required)

Warning: This action cannot be undone. Use with caution.

Open Documentation

Open the official geeup documentation in your default web browser.

geeup readme

Full Documentation

Find the complete documentation at: https://geeup.geetools.xyz

Changelog

Version 2.0.0

• Enhanced path normalization for legacy and cloud projects
• Added service account authentication support
• Improved parallel processing with worker threads
• Added resume and retry capabilities for failed uploads
• Enhanced metadata generation with GDAL integration
• Better error handling and progress tracking
• Rich console output with visual progress bars
• State persistence for resumable workflows

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Acknowledgments

This tool borrows features from another project of mine, geeadd, such as quota estimation, task monitoring, and cancellation capabilities.