Issues Encountered During Local End-to-End Testing of hyphae APIS

[AkarshSahlot]

Issues Encountered During Local End-to-End Testing of hyphae APIS

Summary

This issue documents problems encountered while following the setup guide for local end-to-end testing on both macOS (Docker) and Ubuntu 24.04 (UTM VM). Multiple environment-specific issues prevented successful deployment.

---

Environment Details

Environment 1: macOS with Docker

• OS: macOS
• Docker: Docker Desktop
• Base Image: ubuntu:20.04

Environment 2: Ubuntu VM (UTM)

• OS: Ubuntu 24.04 LTS (Noble) - ARM64
• Virtualization: UTM
• Python: Python 3.12 (default)

---

Issues Encountered

1. Docker Port Mapping Issue (macOS)

Problem: Services running inside Docker container were not accessible from host machine.

Root Cause: Container was started without port mappings using:

docker run -it --name apis-dev ubuntu:20.04

Impact: After running make run, web interfaces at http://localhost:4382 and http://localhost:4390 were unreachable from the host browser.

Solution: Container must be started with explicit port mappings:

docker run -it \
  -p 10000:10000 \
  -p 4382:4382 \
  -p 4390:4390 \
  -p 8000:8000 \
  -p 27017:27017 \
  --name apis-dev \
  ubuntu:20.04

Recommendation: The setup guide should emphasize port mapping in the initial docker run command, not just in the "pre-built image" section.

---

2. MongoDB Installation Failure (Ubuntu 24.04)

Problem: MongoDB package not available in Ubuntu 24.04 repositories.

Error:

Package mongodb is not available, but is referred to by another package.
E: Package 'mongodb' has no installation candidate

Root Cause: Ubuntu 24.04 (Noble) removed the mongodb package from default repositories. Only mongodb-org (official MongoDB repository) is supported.

Impact: Build process fails at dependency installation stage.

Attempted Solution: Install from official MongoDB repository:

# Import MongoDB GPG key
curl -fsSL https://www.mongodb.org/static/pgp/server-7.0.asc | \
   sudo gpg -o /usr/share/keyrings/mongodb-server-7.0.gpg --dearmor

# Add MongoDB repository (ARM64)
echo "deb [ arch=arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list

# Install
sudo apt-get update
sudo apt-get install -y mongodb-org

Recommendation: Update the setup guide to include MongoDB installation instructions for Ubuntu 22.04+ or provide Docker-based MongoDB as an alternative.

---

3. Python Version Incompatibility (Ubuntu 24.04)

Problem: numpy==1.19.4 is incompatible with Python 3.12 (default in Ubuntu 24.04).

Error:

pip._vendor.pyproject_hooks._impl.BackendUnavailable: Cannot import 'setuptools.build_meta'

Root Cause:

• Ubuntu 24.04 ships with Python 3.12
• apis-emulator/requirements.txt specifies numpy==1.19.4
• NumPy 1.19.4 does not support Python 3.12 (released in 2020, predates Python 3.12)

Impact: Build fails at make build-apis-emulator stage.

Current requirements.txt:

wheel==0.30.0
bottle==0.12.18
netifaces==0.10.9
numpy==1.19.4
requests==2.24.0

Attempted Workarounds:

1. Install Python 3.9 (Failed - not available in Ubuntu 24.04 repos)

   sudo apt-get install -y python3.9  # Package not found

2. Update requirements.txt (Untested - requires validation):

   wheel>=0.37.0
   bottle==0.12.25
   netifaces==0.11.0
   numpy==1.24.3
   requests==2.31.0

Recommendation:

• Either pin the Ubuntu version to 20.04/22.04 in the guide
• Or update requirements.txt with Python 3.12-compatible dependencies
• Or use pyenv to manage Python versions

---

4. Missing Docker Command Syntax in Guide

Problem: The guide doesn't clearly explain the difference between starting a fresh container vs. accessing an existing one.

Confusion Point: Users may not realize they need to:

1. Commit the container after setup: docker commit <container_id> hyphae-apis-stable:latest
2. Then run with port mappings
3. Then navigate to APIS directory and run make run

Recommendation: Add a clear workflow diagram showing the container lifecycle.

---

Proposed Updates to Setup Guide

For Docker (macOS/Linux):

1. Update initial docker run command to include port mappings from the start
2. Add troubleshooting section for port mapping issues
3. Clarify the commit workflow - when to commit, when to run

For Ubuntu Native:

1. Specify supported Ubuntu versions (20.04, 22.04 recommended)
2. Add MongoDB installation instructions for Ubuntu 22.04+
3. Add Python version requirements (Python 3.8-3.10 recommended)
4. Provide alternative setup using Docker for MongoDB + pyenv for Python version management

General Improvements:

1. Add a prerequisites checklist at the beginning
2. Add troubleshooting section for common errors
3. Include verification steps after each major installation
4. Provide rollback instructions if setup fails midway

---

Working Solutions Summary

For macOS (Docker):

✅ Use port-mapped container from the start
✅ Commit container after setup to avoid rebuild
✅ Access via http://localhost:4382 from host

For Ubuntu 24.04 (Native):

⚠️ Not fully working - requires dependency updates
❌ MongoDB installation needs workaround
❌ Python 3.12 incompatibility with numpy 1.19.4
🔄 Recommended: Use Ubuntu 20.04 or 22.04 instead

---

Questions for Maintainers

1. Are there plans to update dependencies for Python 3.12 compatibility?
2. Should the guide recommend a specific Ubuntu LTS version?
3. Is there a preferred MongoDB installation method for newer Ubuntu versions?
4. Would a Docker Compose setup be more reliable for cross-platform consistency?

---

Environment Files

If needed, I can provide:

• Full build logs
• System information (uname -a, package versions)
• Modified configuration files