package_spec.md

Pome Package Specification and Runtime Import System

Introduction

This document defines the technical standard for what constitutes a "Pome Package" and details how the C++ Importer class resolves import statements within the Pome programming language. It aims to ensure robustness, usability, and secure handling of dependencies, including native extensions.

0. GitHub Repository Structure for Peck

For Peck to properly discover, install, and manage a Pome package, the package's GitHub repository must adhere to a specific structure, where the repository's root directory is considered the package's root directory.

This means that all package components, including metadata, scripts, native extensions, and assets, must be directly accessible from the top level of the cloned repository.

Key Requirements:

• pome_pkg.json: The package metadata file (pome_pkg.json) must reside in the root directory of the GitHub repository. Peck uses this file to identify the package, its name, version, and dependencies.
• Package Name Consistency: The name field within pome_pkg.json should ideally match the repository name (though Peck uses the name from pome_pkg.json as the canonical identifier).
• Version Tags: For indexed installations, Peck expects Git tags (e.g., v1.0.0) to correspond to specific versions declared in the package index. The packages.json index itself points to specific commit hashes for version integrity.

Example GitHub Repository Layout:

https://github.com/USERNAME/my_package/
├── pome_pkg.json          # REQUIRED: Package metadata at the repository root
├── __init__.pome          # Package entry point
├── module_a.pome
├── sub_package/
│   ├── __init__.pome
│   └── helper.pome
├── lib/                   # Directory for native extensions
│   ├── native_module.so
│   ├── native_module.dylib
│   └── native_module.dll
├── assets/                # Directory for non-code assets
│   ├── config.json
│   └── image.png
├── README.md
└── LICENSE

1. The Package "Anatomy" (File Layout)

A valid Pome package adheres to a strict directory structure.

my_package/
├── __init__.pome      # Package entry point for 'import my_package'
├── module_a.pome      # A Pome script module
├── sub_package/
│   ├── __init__.pome  # Entry for 'import my_package.sub_package'
│   └── helper.pome
├── lib/               # Directory for native extensions (shared libraries)
│   ├── native_module.so    # C++ shared library for Linux
│   ├── native_module.dylib # C++ shared library for macOS
│   └── native_module.dll   # C++ shared library for Windows
├── assets/            # Directory for non-code assets (images, JSON, etc.)
│   ├── config.json
│   └── image.png
└── pome_pkg.json      # Package metadata (version, dependencies, etc.)

Key Points:

• Entry Point Resolution:
  • import my_pkg; will first look for my_pkg.pome.
  • If not found, it will look for my_pkg/__init__.pome within a directory named my_pkg.
  • If neither Pome script is found, and my_pkg is listed as a native module in pome_pkg.json of its parent package, it will attempt to load my_pkg/lib/my_pkg.so (or platform equivalent).
• Native Extensions: Shared libraries for native modules are placed in the lib/ subdirectory within the package folder.
• Asset Handling: Non-code assets (e.g., JSON, images) should reside in the assets/ subdirectory.

2. Package Metadata (pome_pkg.json)

The pome_pkg.json file defines critical metadata for a Pome package. It is located at the root of the package directory.

{
  "name": "my_package",
  "version": "1.0.0",
  "description": "A brief description of my Pome package.",
  "authors": [
    "Author One <author1@example.com>",
    "Author Two"
  ],
  "license": "MIT",
  "pome_version_compatibility": ">=0.1.0 <1.0.0",
  "entry_point": "__init__.pome",
  "modules": [
    "module_a",
    "sub_package"
  ],
  "native_modules": [
    "native_module"
  ],
  "dependencies": {
    "another_package": ">=0.5.0 <1.0.0",
    "utility_lib": "~2.0.0"
  }
}

Field Descriptions:

• name (Required): Unique identifier for the package.
• version (Required): Package version following Semantic Versioning.
• description (Optional): A concise summary of the package's purpose.
• authors (Optional): Array of individuals or organizations contributing to the package.
• license (Optional): License under which the package is distributed (e.g., "MIT").
• pome_version_compatibility (Required): Specifies the compatible Pome interpreter version range (e.g., ">=0.1.0 <1.0.0").
• entry_point (Optional): The primary module file to load for direct package imports (defaults to __init__.pome or [package_name].pome).
• modules (Optional): Array of Pome script modules or sub-packages intended as part of the public API.
• native_modules (Optional): Array of names of native modules (shared libraries) within the package's lib/ directory that are part of the public API.
• dependencies (Optional): An object mapping package names to their compatible version ranges, used by the Peck package manager for dependency resolution.

3. The Import Search Path (Resolution Algorithm)

The Pome::Importer resolves import statements by searching for modules in a defined order, prioritizing local development and virtual environments.

Search Order for findModuleCandidate(logicalPath):

1. Current Working Directory (CWD): The directory where the Pome interpreter was launched.
   • ./
   • ./modules/
2. Virtual Environment (.pome_env): The Importer will traverse up the directory tree from the CWD to find a .pome_env directory. If found, it searches:
   • <project_root>/.pome_env/lib/
3. Environment Variable ($POME_PATH): Paths specified in the POME_PATH environment variable (colon-separated on Unix-like systems, semicolon-separated on Windows).
4. User Home Modules:
   • ~/.pome/modules/
5. System Global Paths: Platform-specific global installation directories.
   • Unix-like: /usr/local/lib/pome/modules/, then /usr/lib/pome/modules/
   • Windows: C:\Program Files\Pome\modules\ (or equivalent Program Files directory)

Pseudocode for Importer::import(logicalPath) and helper findModuleCandidate(logicalPath):

// Helper function to find the candidate file/directory for a given logical path
function findModuleCandidate(logicalPath):
    pathSegments = split logicalPath by '.'
    moduleName = last element of pathSegments

    searchBases = []
    # 1. Current Working Directory
    searchBases.add(currentWorkingDirectory)
    # 2. Current Working Directory Modules
    searchBases.add(currentWorkingDirectory + "/modules")

    # 3. Virtual Environment: Walk up the tree for .pome_env
    currentPath = currentWorkingDirectory
    while currentPath is not root:
        if directoryExists(currentPath + "/.pome_env/lib"):
            searchBases.add(currentPath + "/.pome_env/lib")
            break
        currentPath = parent(currentPath)

    # 4. Environment Variable POME_PATH
    if POME_PATH_env_var is set:
        for path in POME_PATH_env_var.split(':'): // Or ';' for Windows
            if path is not empty:
                searchBases.add(path)

    # 5. User Home Modules
    if HOME_env_var is set:
        searchBases.add(HOME_env_var + "/.pome/modules")

    # 6. System Global Paths
    if OS is Unix:
        searchBases.add("/usr/local/lib/pome/modules")
        searchBases.add("/usr/lib/pome/modules")
    else if OS is Windows:
        searchBases.add(windowsProgramFiles + "/Pome/modules")

    for baseDir in searchBases:
        modulePathSegment = join(pathSegments, "/") // e.g., my_pkg/sub_pkg

        // --- Check for Pome Script Module ---
        // Option A: Single file (e.g., baseDir/my_pkg.pome for 'import my_pkg')
        pomeFileCandidate = baseDir + "/" + join(pathSegments, ".") + ".pome"
        if fileExists(pomeFileCandidate):
            return { type: "POME_SCRIPT_FILE", path: pomeFileCandidate }

        // Option B: Package directory with __init__.pome (e.g., baseDir/my_pkg/__init__.pome for 'import my_pkg')
        initFileCandidate = baseDir + "/" + modulePathSegment + "/__init__.pome"
        if fileExists(initFileCandidate):
            return { type: "POME_PACKAGE_DIR", path: baseDir + "/" + modulePathSegment } // Return package root directory path

        // --- Check for Native Module ---
        // This case handles 'import my_pkg.my_native_module' where my_native_module is a native extension.
        // It requires pome_pkg.json to be present in the parent module's directory.
        // Example: If logicalPath is "my_pkg.sub_native", then pathSegments is ["my_pkg", "sub_native"].
        // moduleDirCandidate becomes baseDir/my_pkg/sub_native.
        // We look for pome_pkg.json in baseDir/my_pkg/
        parentModuleDir = baseDir + "/" + join(pathSegments[0:-1], "/") // baseDir/my_pkg if logicalPath was my_pkg.sub_native
        pkgJsonPath = parentModuleDir + "/pome_pkg.json"

        if fileExists(pkgJsonPath):
            pkgMetadata = parseJson(readFile(pkgJsonPath))
            if moduleName is in pkgMetadata.native_modules: // Check if the last segment is listed as native
                nativeLibCandidate = parentModuleDir + "/lib/" + moduleName + getNativeExtensionSuffix()
                if fileExists(nativeLibCandidate):
                    return { type: "NATIVE_MODULE_FILE", path: nativeLibCandidate }

    return { type: "NOT_FOUND", path: "" }


function Importer::import(logicalPath):
    if isCached(logicalPath):
        return moduleCache_[logicalPath]

    candidate = findModuleCandidate(logicalPath)

    if candidate.type == "NOT_FOUND":
        throw ModuleNotFoundError(logicalPath) // Error handling detailed below

    loadedModule = null
    if candidate.type == "POME_SCRIPT_FILE":
        source = readFile(candidate.path)
        program = parse(source) // Lex, parse, and potentially evaluate
        loadedModule = program // Placeholder for the actual module object
    else if candidate.type == "POME_PACKAGE_DIR":
        // For 'import my_pkg', we evaluate my_pkg/__init__.pome
        source = readFile(candidate.path + "/__init__.pome")
        program = parse(source)
        loadedModule = program // Placeholder
    else if candidate.type == "NATIVE_MODULE_FILE":
        loadedModule = loadNativeModule(candidate.path) // Loads and initializes the C++ shared library
    else:
        // This case should ideally not be reached with proper candidate types
        throw InternalImporterError("Unknown module candidate type encountered during import.")

    moduleCache_[logicalPath] = loadedModule
    return loadedModule

4. Native Loading Specification (ABI Contract)

Pome native modules are implemented as dynamically linked shared libraries that adhere to a specific Application Binary Interface (ABI).

C++ Interface (ABI Contract): A native module must export a C-style function with the following signature:

// In the native module's C++ source file
#include <pome_interpreter.h> // Assuming Pome::Interpreter and Pome::Value definitions

extern "C" Pome::Value PomeInitModule(Pome::Interpreter* interp);

• extern "C": Ensures the function name PomeInitModule is not C++ name-mangled, allowing dynamic linking to find it consistently.
• Pome::Value: The return type. This Pome::Value should typically be a Pome Table object containing the functions, classes, and variables that the native module exposes to the Pome runtime.
• PomeInitModule: The exact function name the Importer will search for.
• Pome::Interpreter* interp: A pointer to the current Pome Interpreter instance, providing the native module access to the Pome runtime for creating Pome values, registering global functions, etc.

Loading Mechanism (loadNativeModule):

function loadNativeModule(libraryPath):
    // Determine platform-specific dynamic loading functions
    // On Unix-like systems (Linux, macOS): dlopen, dlsym, dlclose
    // On Windows: LoadLibraryA, GetProcAddress, FreeLibrary

    handle = dynamic_library_open(libraryPath)
    if handle is null:
        throw NativeModuleError("Failed to load native library: " + libraryPath + ". Error: " + getDynamicLibraryError())

    initFunc = dynamic_library_symbol(handle, "PomeInitModule")
    if initFunc is null:
        dynamic_library_close(handle)
        throw NativeModuleError("Native module " + libraryPath + " does not export 'PomeInitModule' function.")

    // Call the initialization function, passing the current interpreter instance
    moduleTable = initFunc(interpreter_) // interpreter_ is the current Pome interpreter

    // Optionally store 'handle' to prevent premature unloading or for future dynamic unloading.
    // For interpreter lifetime, it's often sufficient to keep it loaded.

    return moduleTable // The Pome::Table object populated by the native module

5. Error Handling

Consistent and informative error messages are crucial for development and debugging.

A. Module Not Found

• Trigger: The findModuleCandidate function fails to locate any matching Pome script file, package directory, or native module file across all defined search paths.

• Error Message:

  ModuleNotFoundError: Module '{logical_path}' not found.
  Searched in:
  - <path_1> (tried <path_1>/<module_file>.pome, <path_1>/<module_dir>/__init__.pome, <path_1>/<parent_module>/lib/<module_name>.<ext>)
  - <path_2> (...)
  ...
  - <path_N> (...)
  

  The detailed list of tried paths helps users understand why a module wasn't found.

B. Cyclic Import Detected

• Trigger: A module attempts to import another module that is currently in the process of being loaded (i.e., it's in the loadingModules_ set).

• Detection Mechanism: The Importer maintains a std::set<std::string> loadingModules_ to track modules whose loading process is in progress. Before starting to load a module, it's added to this set. If an import request for a module already in this set is received, a cyclic import is detected. Once loading is complete (or an error occurs), the module is removed from the set. An RAII helper (ScopedLoadingModule) can manage this.

• Error Message:

  CyclicImportError: Cyclic import detected for module '{logical_path}'.
  Import stack:
  - {module_A} (currently loading)
  - {module_B} (currently loading)
  - {module_C} (attempted to import {logical_path})
  

  Providing the import stack helps pinpoint the exact cycle.

---

This concludes the technical specification for the Pome Package and Runtime Import System.