Merge pull request #28 from colmduff/0.1.5

0.1.5

Author: colmduff

.github/workflows/test.yml

@@ -23,11 +23,8 @@ jobs:
           python -m pip install --upgrade pip
           pip install poetry
 
-      - name: Install dependencies with Poetry
-        run: poetry install --no-interaction --no-root
-
-      - name: Install optigob package (editable)
-        run: poetry install --no-interaction
+      - name: Install dependencies with Poetry (including HiGHS solver)
+        run: poetry install --no-interaction -E solvers
 
       - name: Run tests via Makefile in tests folder
         working-directory: tests

.gitignore

@@ -167,6 +167,14 @@ sankey.py
 #claude
 CLAUDE.md
 .claude_config.yaml
+goals.md
 
-#test logs
+#test files/logs/data
 tests/data/logs/
+.solver_test
+.test_co2e_budget_confirmation.py
+.test_zero_budget_message.py
+.retest.py
+test_bugs_enhanced.py
+sip_bug1.yaml
+sip_bug2.yaml

CHANGELOG.md

@@ -2,6 +2,26 @@
 
 <!--next-version-placeholder-->
 
+## v0.1.5 (17/02/2026)
+
+### Breaking Changes
+
+- **Solver migration**: Replaced `cplex_direct` (previously a core dependency) with **HiGHS** as the recommended solver, now available as an optional extra
+  - Install with `pip install "optigob[solvers]"` or `poetry install -E solvers`
+  - Users without HiGHS can specify any Pyomo-compatible solver via the `solver_name` parameter in their SIP file
+
+### Features
+
+- **Configurable solver**: Users can now specify their preferred solver in the SIP input file (e.g., `solver_name: "highs"`, `solver_name: "glpk"`)
+- **Package-level imports**: Key classes (`Optigob`, `OptiGobDataManager`, `InputHelper`, `configure_logging`, `get_logger`) are now exported from `__init__.py` for cleaner imports (e.g., `from optigob import Optigob`)
+
+### Documentation
+
+- Fixed documentation inconsistencies across README, INSTALLATION, and CLAUDE.md
+- Added `limitations.md` to ReadTheDocs navigation
+- Updated Poetry extras install syntax in installation guide
+- Corrected module tree to reflect actual `substitution/` directory structure
+
 ## v0.1.4 (05/11/2025)
 
 ### Features

INSTALLATION.md

@@ -4,7 +4,7 @@ This guide will help you install and set up the `optigob` package for developmen
 
 ## Requirements
 
-- Python 3.9 or higher
+- Python 3.12 or higher
 - [Poetry](https://python-poetry.org/) (recommended for development)
 - (Optional) [pip](https://pip.pypa.io/) for standard installation
 - (Optional) [conda](https://docs.conda.io/) for environment management
@@ -17,6 +17,17 @@ You can install the latest release from PyPI:
 pip install optigob
 ```
 
+OptiGob requires a solver to run optimizations. The recommended option is
+**HiGHS**, which you can install alongside optigob:
+
+```bash
+pip install "optigob[solvers]"
+```
+
+Alternatively, install just optigob and choose your own Pyomo-compatible solver
+from the list provided below.
+
+
 ## Development Install (Recommended for Contributors)
 
 1. **Clone the repository:**
@@ -32,36 +43,82 @@ pip install optigob
     pip install poetry
     ```
 
-3. **Install dependencies and the package in editable mode:**
+3. **Install dependencies and the project (local-path install):**
 
     ```bash
     poetry install
     ```
 
-4. **Activate the Poetry shell (optional):**
+    Poetry installs the project from your working directory (PEP 660
+    editable-style), so local code changes are reflected immediately without
+    a separate `-e` flag.
+
+    To include the recommended HiGHS solver:
 
     ```bash
-    poetry shell
+    poetry install -E solvers
     ```
 
-5. **Run tests to verify your installation:**
+4. **Run commands in the Poetry environment:**
 
-    ```bash
-    poetry run pytest
-    ```
+    You have two options:
+
+    - **Use `poetry run` prefix** (recommended for single commands):
+      ```bash
+      poetry run pytest
+      poetry run python tests/example.py
+      ```
+
+    - **Or activate the Poetry shell** (for multiple commands in one session):
+      ```bash
+      poetry shell
+      pytest
+      python tests/example.py
+      exit  # to leave the Poetry shell when done
+      ```
 
 ## Optional: Using Conda
 
 If you prefer conda for environment management:
 
 ```bash
-conda create -n optigob python=3.10
+conda create -n optigob python=3.12
 conda activate optigob
 pip install poetry
 poetry install
 ```
 
-## Updating the Package
+To include the recommended HiGHS solver:
+
+```bash
+poetry install -E solvers
+```
+
+## Using Alternative Solvers
+
+OptiGob requires a Pyomo-compatible solver to run optimizations. While **HiGHS** is
+the recommended built-in option (install with `pip install highspy` or
+`poetry install --with solvers`), you can use any solver supported by Pyomo.
+
+To use a different solver, specify the `solver_name` parameter in your configuration:
+
+```python
+from optigob import Optigob
+from optigob.resource_manager.optigob_data_manager import OptiGobDataManager
+
+data = {
+    # ... your scenario parameters ...
+    'solver_name': 'glpk'  # Use GLPK instead of HiGHS
+}
+data_manager = OptiGobDataManager(data)
+optigob = Optigob(data_manager)
+```
+
+For a complete list of available Pyomo-compatible solvers, see the
+[Pyomo documentation](https://pyomo.readthedocs.io/en/stable/getting_started/solvers.html).
+
+To use a specific solver, ensure it is installed in your environment, then
+specify it via `solver_name`.
 
 To update to the latest version from PyPI:
 
@@ -82,7 +139,7 @@ poetry update
 
 ## Additional Resources
 
-- [OptiGob Documentation](https://github.com/colmduff/OptiGob#readme)
+- [OptiGob Documentation](https://optigob.readthedocs.io/en/latest/)
 - [Poetry Documentation](https://python-poetry.org/docs/)
 
 If you have any questions or need help, please open an issue on GitHub or contact the maintainers.

README.md

@@ -8,13 +8,13 @@
 [GitHub](https://github.com/colmduff/OptiGob) |
 [Documentation](https://optigob.readthedocs.io/en/latest/) |
 
-A land use change and environmental assessment tool based on preconfigured data animal population numbers based on negative emissions allowance.
+`OptiGob` is a Python-based decision support tool for exploring Ireland's agriculture, forestry, and other land use (AFOLU) sector under different climate and land use transition pathways. It combines pre-computed scenario data from [GOBLIN Lite](https://github.com/colmduff/goblin_lite) (agriculture and land use), FERS-CBM (forestry carbon balance), and LCAD2.0 (anaerobic digestion) into a single interface that optimises livestock populations subject to land and emissions constraints — without requiring users to run the upstream modelling chains. It is designed for researchers, policymakers, and educators investigating environmental and economic trade-offs associated with net-zero and split-gas emissions targets.
 
 ---
 
 ## High-Level Architecture & Module Interaction
 
-The `optigob` package is a modular land use and environmental assessment framework. It is designed to calculate, aggregate, and analyze greenhouse gas emissions, land use, protein, bioenergy, harvested wood products, and substitution impacts for both baseline and scenario cases. The system is built around a central API (`Optigob`), which coordinates specialized modules for each sector and data type.
+The `optigob` package is a modular land use and environmental assessment framework. It calculates greenhouse gas emissions, land use, protein output, bioenergy, harvested wood products, and substitution impacts for both baseline and scenario cases. The system is built around a central API (`Optigob`), which coordinates specialised modules for each sector and data type.
 
 ### Module Structure
 
@@ -24,7 +24,8 @@ optigob/
 │   ├── baseline_emissions.py
 │   ├── emissions_budget.py
 │   ├── landarea_budget.py
-│   ├── econ_output.py
+│   └── econ_output.py
+├── substitution/
 │   └── substitution.py
 ├── bioenergy/
 │   └── bioenergy_budget.py
@@ -54,7 +55,7 @@ optigob/
 - **budget_model/baseline_emissions.py**: Provides baseline emissions by sector for all gases.
 - **budget_model/landarea_budget.py**: Calculates land area (aggregated, disaggregated, HNV) by sector for baseline and scenario.
 - **budget_model/econ_output.py**: Calculates protein, bioenergy, harvested wood products, and livestock population by sector.
-- **budget_model/substitution.py**: Centralizes logic for substitution impacts (e.g., wood for fossil, protein crop substitution).
+- **substitution/substitution.py**: Centralizes logic for substitution impacts (e.g., wood for fossil, protein crop substitution).
 - **protein_crops/protein_crops_budget.py**: Handles protein crop area, yield, and protein output calculations.
 - **livestock/livestock_budget.py**: Calculates livestock sector budgets, including emissions, land use, and protein.
 - **forest/forest_budget.py**: Handles forest sector land area, emissions, and harvested wood product calculations.
@@ -90,9 +91,56 @@ optigob/
 - Substitution logic is centralized in `substitution.py` and called by emissions and output modules as needed.
 - Specialized sector modules (livestock, forest, protein crops, etc.) encapsulate sector-specific calculations and are used by the budget modules.
 
-### Default Database
+### Input Data and Default Database
 
-A default database offers multiple permutations of abatement and productivity scenarios, but certain parameters (e.g., wetland restoration, forestry on organic soils) are limited to predefined combinations. A helper function identifies valid options to prevent unsupported configurations. Custom databases can also be supplied, provided they follow the expected schema.
+`optigob` ships with a pre-built SQLite database (`src/optigob/database/optigob_default_0.1.1.db`) that is installed automatically with the package. **No additional data download is required.**
+
+#### Data provenance
+
+The bundled database does not contain raw observational data. It contains **derived scenario scaler tables** generated by three upstream, sector-specific models:
+
+| Upstream model | Domain | Reference |
+|---|---|---|
+| [GOBLIN Lite](https://github.com/colmduff/goblin_lite) | Agriculture and land use emissions, areas, livestock productivity | Duffy et al. (2022, 2024) |
+| [FERS-CBM](https://www.ucd.ie/forestecosystemresearch/) | Forest carbon balance (afforestation, existing forest, harvested wood) | Black et al. (2025); Kurz et al. (2008) |
+| [LCAD2.0](https://github.com/GOBLIN-Proj/lcad_gob) | Anaerobic digestion area, emissions, and bioenergy output | Martinez-Acre et al. (2025) |
+
+These models were run across a defined set of valid parameter combinations for **Ireland**, and reflect three levels of abatement (Baseline, MACC, Frontier). The resulting scaler tables were stored in the SQLite database. At runtime, `optigob` queries these scalers to parameterise its optimisation model — effectively acting as a surrogate that stitches together pre-computed sectoral outputs without re-running the upstream models.
+
+#### Database contents
+
+The database contains scaler tables for:
+
+- Livestock emissions, area, and protein (by species, abatement type, productivity tier)
+- Forest emissions, area, harvested wood products, and wood CCS
+- Organic soil emissions and area
+- Anaerobic digestion emissions and area
+- Protein crop emissions, area, and protein output
+- Willow bioenergy area and output
+- Substitution impacts (wood-for-fossil, crop-for-animal protein)
+
+#### Predefined combinations
+
+Not all parameter combinations are valid — only those that were modelled by the upstream tools exist in the database. Use the `InputHelper` class to explore valid combinations:
+
+```python
+from optigob.input_helper import InputHelper
+helper = InputHelper()
+helper.print_all_combos()  # View all valid parameter sets
+```
+
+See `INPUT_VARIABLES.md` for full parameter definitions and validation requirements.
+
+#### Using a custom database
+
+While the default database is supplied, users can update it or supply a custom database using `DatabaseManager`:
+
+```python
+from optigob.resource_manager.database_manager import DatabaseManager
+db = DatabaseManager(database_path="/path/to/custom_database.db")
+```
+
+The custom database must follow the same table schema as the default.
 
 ---
 
@@ -109,15 +157,30 @@ A default database offers multiple permutations of abatement and productivity sc
 
 To install the package, use pip:
 
+```bash
+pip install "optigob[solvers]"
+```
+
+if you would prefer to use an alternative solver to HiGHS, the install command is: 
+
 ```bash
 pip install optigob
 ```
 
+Be sure to specify the solver in "solver_name" in the SIP.yaml.
+
+For example: 
+
+```yaml
+solver_name: "highs"
+```
+
 ## Usage
 
 Here is some example input data:
 
 ```yaml
+    solver_name: "highs"
     AR: 5
     split_gas: true
     split_gas_frac: 0.3

docs/conf.py

@@ -7,7 +7,7 @@
 # -- Project information -----------------------------------------------------
 
 project = u"optigob"
-copyright = u"2025, Colm Duffy"
+copyright = u"2025-2026, Colm Duffy"
 author = u"Colm Duffy"
 
 # -- General configuration ---------------------------------------------------

docs/data/sip.yaml

@@ -1,3 +1,4 @@
+solver_name: "highs"
 AR: 5
 split_gas: true
 split_gas_frac: 0.3

docs/example.ipynb

@@ -11,7 +11,7 @@
         },
         {
             "cell_type": "code",
-            "execution_count": 16,
+            "execution_count": null,
             "metadata": {},
             "outputs": [
                 {
@@ -53,9 +53,9 @@
                 }
             ],
             "source": [
-                "from optigob.optigob import Optigob\n",
-                "from optigob.resource_manager.optigob_data_manager import OptiGobDataManager\n",
-                "from optigob.input_helper import InputHelper\n",
+                "from optigob import Optigob\n",
+                "from optigob import OptiGobDataManager\n",
+                "from optigob import InputHelper\n",
                 "\n",
                 "print(\"OptiGob Budget Model Input Combinations\")\n",
                 "\n",

docs/index.md

@@ -11,5 +11,6 @@ contributing.md
 conduct.md
 installation.md
 input_variables.md
+limitations.md
 autoapi/index
 ```