Add HSEvo method and task/example scripts

README.md

@@ -176,6 +176,7 @@ Check [GUI Introduction](https://llm4ad-doc.readthedocs.io/en/latest/getting_sta
 | Methods                           | Paper title                                                  |
 | --------------------------------- | ------------------------------------------------------------ |
 | **EoH**                           | [Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model](https://openreview.net/pdf?id=BwAkaxqiLB) (ICML 2024)<br>[Algorithm Evolution using Large Language Model](https://arxiv.org/abs/2311.15249) (Arxiv 2023, AEL, the early version of EoH) |
+| **HSEvo**                         | [HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMs](https://arxiv.org/abs/2412.14995) (AAAI 2025) |
 | **MEoH**                          | [Multi-objective Evolution of Heuristic Using Large Language Model](https://arxiv.org/abs/2409.16867) (AAAI 25) |
 | **FunSearch**                     | [Mathematical Discoveries from Program Search with Large Language Models](https://www.nature.com/articles/s41586-023-06924-6) (Nature 2024) |
 | **(1+1)-EPS** <br/>(HillClimbing) | [Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models](https://arxiv.org/abs/2407.10873) (PPSN 2024) |

example/methods/hsevo/README.md

@@ -0,0 +1,133 @@
+# HSEvo: Diversity-Driven Harmony Search + Genetic Algorithm
+
+**HSEvo** (*Harmony Search Evolution*) is an LLM-based evolutionary program search method that combines genetic operators with a **harmony search (HS)** operator for automatic heuristic design. It is integrated into [LLM4AD](https://github.com/Optima-CityU/llm4ad) as `llm4ad.method.hsevo`.
+
+**Paper:** [HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMs](https://doi.org/10.1609/aaai.v39i25.34898) (AAAI 2025)
+
+**Upstream code:** [datphamvn/HSEvo](https://github.com/datphamvn/hsevo)
+
+---
+
+## Overview
+
+Each HSEvo generation runs:
+
+1. **Selection** — random parent pairs with different objective values
+2. **Reflection** — flash + comprehensive reflection prompts on the population
+3. **Crossover** — LLM combines two parents into a new heuristic
+4. **Mutation** — LLM mutates the elitist using reflection hints
+5. **Harmony search** — pick one untuned individual, ask the LLM to expose tunable numeric parameters, then search parameter space with classical HS (`hmcr`, `par`, `bandwidth`)
+
+The HS step is the distinctive operator: the LLM rewrites hardcoded thresholds/weights as function defaults, defines `parameter_ranges`, and HSEvo evaluates multiple parameter settings on the **same** heuristic structure.
+
+Successful HS runs log `[HS-CHECK]` with `distinct_init_objs > 1`, confirming different parameter vectors produced different scores.
+
+---
+
+## Quick Start
+
+Configure your LLM API, then run any task script:
+
+```bash
+uv run python example/tasks/online_bin_packing/run_hsevo.py
+```
+
+Minimal Python example:
+
+```python
+from llm4ad.task.optimization.online_bin_packing import OBPEvaluation
+from llm4ad.tools.llm.llm_api_https import HttpsApi
+from llm4ad.method.hsevo import HSEvo, HSEvoProfiler
+
+llm = HttpsApi(host='xxx', key='sk-xxx', model='xxx', timeout=60)
+task = OBPEvaluation()
+
+method = HSEvo(
+    llm=llm,
+    profiler=HSEvoProfiler(log_dir='logs/hsevo', log_style='simple'),
+    evaluation=task,
+    max_sample_nums=100,
+    pop_size=4,
+    init_pop_size=10,
+    mutation_rate=0.5,
+    hm_size=5,
+    hmcr=0.7,
+    par=0.5,
+    bandwidth=0.2,
+    max_iter=5,
+    num_samplers=4,
+    num_evaluators=4,
+)
+
+method.run()
+```
+
+A copy of this OBP script lives at [`run_hsevo_obp.py`](./run_hsevo_obp.py).
+
+---
+
+## Hyper-parameters
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `max_sample_nums` | `450` | Stop after this many function evaluations |
+| `pop_size` | `10` | Parents used for crossover each generation |
+| `init_pop_size` | `30` | Initial population size (rotating scientist personas) |
+| `mutation_rate` | `0.5` | Fraction of `pop_size` mutated from the elitist |
+| `hm_size` | `5` | Harmony memory size (initial random parameter vectors) |
+| `hmcr` | `0.7` | Harmony memory considering rate |
+| `par` | `0.5` | Pitch adjustment rate |
+| `bandwidth` | `0.2` | Pitch adjustment bandwidth (fraction of each param range) |
+| `max_iter` | `5` | HS improvisation iterations after memory init |
+| `num_samplers` | `4` | Parallel LLM sampling threads |
+| `num_evaluators` | `4` | Parallel evaluation workers |
+
+Defaults are also listed in [`llm4ad/method/hsevo/paras.yaml`](../../../llm4ad/method/hsevo/paras.yaml).
+
+---
+
+## Task entry scripts
+
+Every task under [`example/tasks/`](../../tasks/) that has `run_eoh.py` also provides `run_hsevo.py`:
+
+| Task | Script |
+|------|--------|
+| Online bin packing | [`example/tasks/online_bin_packing/run_hsevo.py`](../../tasks/online_bin_packing/run_hsevo.py) |
+| TSP constructive | [`example/tasks/tsp_construct/run_hsevo.py`](../../tasks/tsp_construct/run_hsevo.py) |
+| CVRP constructive | [`example/tasks/cvrp_construct/run_hsevo.py`](../../tasks/cvrp_construct/run_hsevo.py) |
+| QAP | [`example/tasks/qap/run_hsevo.py`](../../tasks/qap/run_hsevo.py) |
+| FSSP / JSSP | [`example/tasks/fssp_construct/run_hsevo.py`](../../tasks/fssp_construct/run_hsevo.py) |
+| Orienteering | [`example/tasks/orienteering_construct/run_hsevo.py`](../../tasks/orienteering_construct/run_hsevo.py) |
+| VRPTW | [`example/tasks/vrptw_construct/run_hsevo.py`](../../tasks/vrptw_construct/run_hsevo.py) |
+| Pymoo MOEA/D | [`example/tasks/pymoo_moead/run_hsevo.py`](../../tasks/pymoo_moead/run_hsevo.py) |
+| Car racing control | [`example/tasks/control_carracing/run_hsevo.py`](../../tasks/control_carracing/run_hsevo.py) |
+| Moon lander control | [`example/tasks/control_moonlander/run_hsevo.py`](../../tasks/control_moonlander/run_hsevo.py) |
+| Circle packing | [`example/tasks/circle_packing/EoH_settings&logs/run_hsevo.py`](../../tasks/circle_packing/EoH_settings&logs/run_hsevo.py) |
+
+---
+
+## Logging
+
+`HSEvoProfiler` writes:
+
+- `run_log.txt` — generation progress, `[HS-CHECK]`, `harmony_search: OK/FAILED`
+- `samples/` — evaluated programs tagged by `operator` (`init`, `crossover`, `mutation`, `harmony_search`, …)
+- `population/` — per-generation population checkpoints
+
+Profiler variants: `HSEvoTensorboardProfiler`, `HSEvoWandbProfiler`.
+
+---
+
+## Citation
+
+```bibtex
+@inproceedings{dat2025hsevo,
+  title={Hsevo: Elevating automatic heuristic design with diversity-driven harmony search and genetic algorithm using llms},
+  author={Dat, Pham Vu Tuan and Doan, Long and Binh, Huynh Thi Thanh},
+  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
+  volume={39},
+  number={25},
+  pages={26931--26938},
+  year={2025}
+}
+```

example/methods/hsevo/run_hsevo_obp.py

@@ -0,0 +1,45 @@
+from __future__ import annotations
+
+import os
+import sys
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../..")))
+
+from llm4ad.task.optimization.online_bin_packing import OBPEvaluation
+from llm4ad.tools.llm.llm_api_https import HttpsApi
+from llm4ad.method.hsevo import HSEvo, HSEvoProfiler
+
+
+def main():
+    llm = HttpsApi(
+        host="xxx",  # your host endpoint, e.g., 'api.openai.com', 'api.deepseek.com'
+        key="sk-xxx",  # your key, e.g., 'sk-abcdefghijklmn'
+        model="xxx",  # your llm, e.g., 'gpt-3.5-turbo'
+        timeout=60,
+    )
+
+    task = OBPEvaluation()
+
+    method = HSEvo(
+        llm=llm,
+        profiler=HSEvoProfiler(log_dir="logs/hsevo", log_style="simple"),
+        evaluation=task,
+        max_sample_nums=100,
+        pop_size=4,
+        init_pop_size=10,
+        mutation_rate=0.5,
+        hm_size=5,
+        hmcr=0.7,
+        par=0.5,
+        bandwidth=0.2,
+        max_iter=5,
+        num_samplers=4,
+        num_evaluators=4,
+        debug_mode=False,
+    )
+
+    method.run()
+
+
+if __name__ == "__main__":
+    main()

example/tasks/circle_packing/EoH_settings&logs/run_hsevo.py

@@ -0,0 +1,45 @@
+from __future__ import annotations
+
+import os
+import sys
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../..")))
+
+from evaluation import CirclePackingEvaluation
+from llm4ad.tools.llm.llm_api_https import HttpsApi
+from llm4ad.method.hsevo import HSEvo, HSEvoProfiler
+
+
+def main():
+    llm = HttpsApi(
+        host="xxx",  # your host endpoint, e.g., 'api.openai.com', 'api.deepseek.com'
+        key="sk-xxx",  # your key, e.g., 'sk-abcdefghijklmn'
+        model="xxx",  # your llm, e.g., 'gpt-3.5-turbo'
+        timeout=120,
+    )
+
+    task = CirclePackingEvaluation(timeout_seconds=1200)
+
+    method = HSEvo(
+        llm=llm,
+        profiler=HSEvoProfiler(log_dir="logs/hsevo", log_style="simple"),
+        evaluation=task,
+        max_sample_nums=100,
+        pop_size=4,
+        init_pop_size=10,
+        mutation_rate=0.5,
+        hm_size=5,
+        hmcr=0.7,
+        par=0.5,
+        bandwidth=0.2,
+        max_iter=5,
+        num_samplers=4,
+        num_evaluators=4,
+        debug_mode=False,
+    )
+
+    method.run()
+
+
+if __name__ == "__main__":
+    main()

example/tasks/control_carracing/run_hsevo.py

@@ -0,0 +1,56 @@
+from __future__ import annotations
+
+import os
+import sys
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../..")))
+
+from llm4ad.task.machine_learning.car_racing import RacingCarEvaluation
+from llm4ad.tools.llm.llm_api_https import HttpsApi
+from llm4ad.method.hsevo import HSEvo, HSEvoProfiler
+
+
+def main():
+    llm = HttpsApi(
+        host="xxx",  # your host endpoint, e.g., 'api.openai.com', 'api.deepseek.com'
+        key="sk-xxx",  # your key, e.g., 'sk-abcdefghijklmn'
+        model="xxx",  # your llm, e.g., 'gpt-3.5-turbo'
+        timeout=120,
+    )
+
+    seeds = [1]
+    instance_set = {idx: seed for idx, seed in enumerate(seeds)}
+    using_seeds = list(range(10, 20))
+    ins_to_be_solve_set = {idx: seed for idx, seed in enumerate(using_seeds)}
+
+    task = RacingCarEvaluation(
+        whocall="eoh",
+        run_mode="Training",
+        instance_set=instance_set,
+        ins_to_be_solve_set=ins_to_be_solve_set,
+        objective_value=100,
+    )
+
+    method = HSEvo(
+        llm=llm,
+        profiler=HSEvoProfiler(log_dir="logs/hsevo", log_style="simple"),
+        evaluation=task,
+        max_sample_nums=100,
+        pop_size=4,
+        init_pop_size=10,
+        mutation_rate=0.5,
+        hm_size=5,
+        hmcr=0.7,
+        par=0.5,
+        bandwidth=0.2,
+        max_iter=5,
+        num_samplers=4,
+        num_evaluators=4,
+        debug_mode=False,
+    )
+
+    method.run()
+
+
+if __name__ == "__main__":
+    main()

example/tasks/control_moonlander/run_hsevo.py

@@ -0,0 +1,96 @@
+from __future__ import annotations
+
+import os
+import sys
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../..")))
+
+from llm4ad.task.machine_learning.moon_lander import (
+    MoonLanderEvaluation,
+    moon_lander_feature,
+)
+from llm4ad.tools.llm.llm_api_https import HttpsApi
+from llm4ad.method.hsevo import HSEvo, HSEvoProfiler
+
+
+def main():
+    llm = HttpsApi(
+        host="xxx",  # your host endpoint, e.g., 'api.openai.com', 'api.deepseek.com'
+        key="sk-xxx",  # your key, e.g., 'sk-abcdefghijklmn'
+        model="xxx",  # your llm, e.g., 'gpt-3.5-turbo'
+        timeout=120,
+    )
+
+    seeds = [
+        6,
+        9,
+        17,
+        29,
+        57,
+        44,
+        18,
+        69,
+        26,
+        68,
+        65,
+        23,
+        51,
+        93,
+        16,
+        87,
+        92,
+        90,
+        22,
+        73,
+        60,
+        10,
+        19,
+        97,
+        11,
+        14,
+        99,
+        98,
+        8,
+        28,
+        43,
+        56,
+        89,
+        15,
+        74,
+    ]
+    instance_set = {idx: seed for idx, seed in enumerate(seeds)}
+    using_seeds = list(range(100, 150))
+    ins_to_be_solve_set = {idx: seed for idx, seed in enumerate(using_seeds)}
+
+    task = MoonLanderEvaluation(
+        whocall="eoh",
+        instance_set=instance_set,
+        run_mode="Training",
+        ins_to_be_solve_set=ins_to_be_solve_set,
+        feature_pipeline=moon_lander_feature,
+        objective_value=230,
+    )
+
+    method = HSEvo(
+        llm=llm,
+        profiler=HSEvoProfiler(log_dir="logs/hsevo", log_style="simple"),
+        evaluation=task,
+        max_sample_nums=100,
+        pop_size=4,
+        init_pop_size=10,
+        mutation_rate=0.5,
+        hm_size=5,
+        hmcr=0.7,
+        par=0.5,
+        bandwidth=0.2,
+        max_iter=5,
+        num_samplers=4,
+        num_evaluators=4,
+        debug_mode=False,
+    )
+
+    method.run()
+
+
+if __name__ == "__main__":
+    main()