Skip to content

Rename LLaMEA-SAGE to LLaMEA-CFG throughout codebase #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nikivanstein merged 1 commit into from
Aug 21, 2026
Merged

Rename LLaMEA-SAGE to LLaMEA-CFG throughout codebase #164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<h1 align="center">LLaMEA-SAGE: Guiding Automated Algorithm Design with
<h1 align="center">LLaMEA-CFG: Guiding Automated Algorithm Design with
Structural Feedback from Explainable AI</h1>


## Introduction
This is the reproducability repository for the paper "LLaMEA-SAGE: Guiding Automated Algorithm Design with Structural Feedback from Explainable AI".
This is the reproducability repository for the paper "LLaMEA-CFG: Guiding Automated Algorithm Design with Structural Feedback from Explainable AI".



Expand Down Expand Up @@ -44,14 +44,14 @@ make sure you have `uv` installed.
```bash
uv run python run_archive_guided.py
```
This will run Experiment 1 (LLaMEA and LLaMEA-SAGE on SBOX-COST).
This will run Experiment 1 (LLaMEA and LLaMEA-CFG on SBOX-COST).

```bash
uv run python run_archive_guided-2.py
uv run python run_archive_guided-2-1.py
uv run python run_archive_guided-2-2.py
```
This will run Experiment 2 (LLaMEA and LLaMEA-SAGE on MA-BBOB), baselines and the abblation using Gemini respectively.
This will run Experiment 2 (LLaMEA and LLaMEA-CFG on MA-BBOB), baselines and the abblation using Gemini respectively.

3. Produce paper artifacts.

Expand Down
4 changes: 2 additions & 2 deletions examples/visualize_kerneltuner.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@
"\n",
"token_df = token_df1\n",
"\n",
"#token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE', 'LHNS:vns': \"LHNS\"}, inplace=True)\n",
"#token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG', 'LHNS:vns': \"LHNS\"}, inplace=True)\n",
"\n",
"summary = (\n",
" token_df.groupby([\"problem_name\", \"method_name\"])[\"tokens\"].mean().reset_index()\n",
Expand Down Expand Up @@ -190,7 +190,7 @@
"\n",
"token_df = token_df1\n",
"\n",
"# token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE', 'LHNS:vns': \"LHNS\"}, inplace=True)\n",
"# token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG', 'LHNS:vns': \"LHNS\"}, inplace=True)\n",
"\n",
"summary = (\n",
" token_df.groupby([\"problem_name\", \"method_name\"])[\"tokens\"]\n",
Expand Down
6 changes: 3 additions & 3 deletions final_validation.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,7 +425,7 @@
" if label == \"ES\":\n",
" handle.set_label(\"LLaMEA\")\n",
" if label == \"ES-guided\":\n",
" handle.set_label(\"LLaMEA-SAGE\")\n",
" handle.set_label(\"LLaMEA-CFG\")\n",
" if label == \"LHNS:vns\":\n",
" handle.set_label(\"LHNS\")\n",
"\n",
Expand Down Expand Up @@ -502,7 +502,7 @@
" if label == \"ES\":\n",
" handle.set_label(\"LLaMEA\")\n",
" if label == \"ES-guided\":\n",
" handle.set_label(\"LLaMEA-SAGE\")\n",
" handle.set_label(\"LLaMEA-CFG\")\n",
" if label == \"LHNS:vns\":\n",
" handle.set_label(\"LHNS\")\n",
"\n",
Expand Down Expand Up @@ -581,7 +581,7 @@
" if label == \"ES\":\n",
" handle.set_label(\"LLaMEA\")\n",
" if label == \"ES-guided\":\n",
" handle.set_label(\"LLaMEA-SAGE\")\n",
" handle.set_label(\"LLaMEA-CFG\")\n",
" if label == \"LHNS:vns\":\n",
" handle.set_label(\"LHNS\")\n",
"\n",
Expand Down
4 changes: 2 additions & 2 deletions iohblade/plots.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,9 +452,9 @@ def plot_experiment_CEG(
if method == "ES":
method_title = "LLaMEA"
elif method == "ES-guided":
method_title = "LLaMEA-SAGE"
method_title = "LLaMEA-CFG"
elif method == "ES-guided-new":
method_title = "LLaMEA-SAGE"
method_title = "LLaMEA-CFG"
ax.set_title(f"{method_title} run:{seed}")
if seed_i > 0:
ax.set_ylabel(None)
Expand Down
2 changes: 1 addition & 1 deletion run_archive_guided-2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
]

LLaMEA_1 = LLaMEA(llm, budget=budget, name="LLaMEA", mutation_prompts=mutation_prompts, n_parents=4, n_offspring=16, elitism=True)
LLaMEA_2 = LLaMEA(llm, budget=budget, name="LLaMEA-SAGE", mutation_prompts=mutation_prompts, n_parents=4, n_offspring=16, elitism=True, feature_guided_mutation=True, parent_selection="tournament", tournament_size=2)
LLaMEA_2 = LLaMEA(llm, budget=budget, name="LLaMEA-CFG", mutation_prompts=mutation_prompts, n_parents=4, n_offspring=16, elitism=True, feature_guided_mutation=True, parent_selection="tournament", tournament_size=2)


methods = [LLaMEA_1, LLaMEA_2]
Expand Down
2 changes: 1 addition & 1 deletion run_archive_guided-3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
]

LLaMEA_1 = LLaMEA(llm, budget=budget, name="LLaMEA", mutation_prompts=mutation_prompts, n_parents=4, n_offspring=16, elitism=True)
LLaMEA_2 = LLaMEA(llm, budget=budget, name="LLaMEA-SAGE", mutation_prompts=mutation_prompts, n_parents=4, n_offspring=16, elitism=True, feature_guided_mutation=True, parent_selection="tournament", tournament_size=2)
LLaMEA_2 = LLaMEA(llm, budget=budget, name="LLaMEA-CFG", mutation_prompts=mutation_prompts, n_parents=4, n_offspring=16, elitism=True, feature_guided_mutation=True, parent_selection="tournament", tournament_size=2)


methods = [LLaMEA_1, LLaMEA_2]
Expand Down
2 changes: 1 addition & 1 deletion run_archive_guided.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
]

LLaMEA_1 = LLaMEA(llm, budget=budget, name="LLaMEA", mutation_prompts=mutation_prompts, n_parents=8, n_offspring=8, elitism=True)
LLaMEA_2 = LLaMEA(llm, budget=budget, name="LLaMEA-SAGE", mutation_prompts=mutation_prompts, n_parents=8, n_offspring=8, elitism=True, feature_guided_mutation=True)
LLaMEA_2 = LLaMEA(llm, budget=budget, name="LLaMEA-CFG", mutation_prompts=mutation_prompts, n_parents=8, n_offspring=8, elitism=True, feature_guided_mutation=True)

methods = [LLaMEA_1, LLaMEA_2]

Expand Down
4 changes: 2 additions & 2 deletions visualize_1.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -666,7 +666,7 @@
"new_labels = []\n",
"for label in previous_labels:\n",
" if label == \"ES-guided-new\":\n",
" new_labels.append(\"LLaMEA-SAGE\")\n",
" new_labels.append(\"LLaMEA-CFG\")\n",
" elif label == \"ES\":\n",
" new_labels.append(\"LLaMEA\")\n",
" else:\n",
Expand Down Expand Up @@ -704,7 +704,7 @@
"new_labels = []\n",
"for label in previous_labels:\n",
" if label == \"ES-guided-new vs ES\":\n",
" new_labels.append(\"LLaMEA-SAGE vs LLaMEA\")\n",
" new_labels.append(\"LLaMEA-CFG vs LLaMEA\")\n",
" else:\n",
" new_labels.append(label)\n",
"ax.legend(handles=handles, labels=new_labels)\n",
Expand Down
24 changes: 12 additions & 12 deletions visualize_2.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"['LLaMEA', 'LLaMEA-SAGE', 'MCTS_AHD', 'ES-guided', 'LHNS:vns', 'ES']\n",
"['LLaMEA', 'LLaMEA-CFG', 'MCTS_AHD', 'ES-guided', 'LHNS:vns', 'ES']\n",
"['MA_BBOB']\n"
]
}
Expand All @@ -210,7 +210,7 @@
"For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.\n",
"\n",
"\n",
" token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE', 'LHNS:vns': \"LHNS\"}, inplace=True)\n"
" token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG', 'LHNS:vns': \"LHNS\"}, inplace=True)\n"
]
}
],
Expand All @@ -226,7 +226,7 @@
"\n",
"token_df = pd.concat([token_df1, token_df2, token_df3], axis=0)\n",
"\n",
"token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE', 'LHNS:vns': \"LHNS\"}, inplace=True)\n",
"token_df['method_name'].replace({'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG', 'LHNS:vns': \"LHNS\"}, inplace=True)\n",
"\n",
"\n"
]
Expand Down Expand Up @@ -261,7 +261,7 @@
"source": [
"\n",
"filtered = token_df[\n",
" token_df[\"method_name\"].isin([\"LLaMEA\", \"LLaMEA-SAGE\", \"LHNS\", \"MCTS_AHD\"])\n",
" token_df[\"method_name\"].isin([\"LLaMEA\", \"LLaMEA-CFG\", \"LHNS\", \"MCTS_AHD\"])\n",
"]\n",
"\n",
"\n",
Expand Down Expand Up @@ -313,7 +313,7 @@
}
],
"source": [
"fig = plot_convergence(logger, metric=\"AOCC\", aggregation=\"mean\", methods=[\"LLaMEA\", \"LLaMEA-SAGE\", \"LHNS:vns\",\"MCTS_AHD\"], replace_labels={'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE'}, save=False, return_fig=True, budget=200, separate_lines=False, variance_aggregation=\"sem\")\n",
"fig = plot_convergence(logger, metric=\"AOCC\", aggregation=\"mean\", methods=[\"LLaMEA\", \"LLaMEA-CFG\", \"LHNS:vns\",\"MCTS_AHD\"], replace_labels={'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG'}, save=False, return_fig=True, budget=200, separate_lines=False, variance_aggregation=\"sem\")\n",
"ax = fig.axes[0] # or however you're getting your axis\n",
"fig.set_size_inches(6, 5)\n",
"handles, previous_labels = ax.get_legend_handles_labels()\n",
Expand All @@ -322,7 +322,7 @@
" if label == \"ES\":\n",
" new_labels.append(\"LLaMEA\")\n",
" elif label == \"ES-guided\":\n",
" new_labels.append(\"LLaMEA-SAGE\")\n",
" new_labels.append(\"LLaMEA-CFG\")\n",
" elif label == \"LHNS:vns\":\n",
" new_labels.append(\"LHNS\")\n",
" else:\n",
Expand Down Expand Up @@ -367,15 +367,15 @@
}
],
"source": [
"fig = plot_speedup(logger, method_fast=\"LLaMEA-SAGE\", method_slow=\"LLaMEA\", save=False, return_fig=True, budget=200, replace_labels={'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE'})\n",
"fig = plot_speedup(logger, method_fast=\"LLaMEA-CFG\", method_slow=\"LLaMEA\", save=False, return_fig=True, budget=200, replace_labels={'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG'})\n",

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep raw method keys when plotting archived results

When this notebook is run against the bundled paper_results/MABBOB_guided* logs, the raw methods are still ES, ES-guided, LLaMEA, LLaMEA-SAGE, LHNS:vns, and MCTS_AHD; there is no raw LLaMEA-CFG entry. plot_speedup validates method_fast against logger.get_methods_problems() before applying replace_labels, so this renamed lookup now raises ValueError and stops generation of MABBOB-speedup.png rather than just changing the displayed label. Keep the lookup key as LLaMEA-SAGE (or rename/map the archived logs before the existence check) and use LLaMEA-CFG only as the display label.

Useful? React with 👍 / 👎.

"ax = fig.axes[0] # or however you're getting your axis\n",
"fig.set_size_inches(6, 5)\n",
"ax.set_title(None)\n",
"handles, previous_labels = ax.get_legend_handles_labels()\n",
"new_labels = []\n",
"for label in previous_labels:\n",
" if label == \"ES-guided vs ES\":\n",
" new_labels.append(\"LLaMEA-SAGE vs LLaMEA\")\n",
" new_labels.append(\"LLaMEA-CFG vs LLaMEA\")\n",
" else:\n",
" new_labels.append(label)\n",
"ax.legend(handles=handles, labels=new_labels)\n",
Expand Down Expand Up @@ -447,7 +447,7 @@
" <tr>\n",
" <th>0</th>\n",
" <td>MA_BBOB</td>\n",
" <td>LLaMEA-SAGE</td>\n",
" <td>LLaMEA-CFG</td>\n",
" <td>MCTS_AHD</td>\n",
" <td>5</td>\n",
" <td>144.165807</td>\n",
Expand All @@ -469,7 +469,7 @@
],
"text/plain": [
" problem method_a method_b n_seeds mean_auc_a mean_auc_b \\\n",
"0 MA_BBOB LLaMEA-SAGE MCTS_AHD 5 144.165807 112.480177 \n",
"0 MA_BBOB LLaMEA-CFG MCTS_AHD 5 144.165807 112.480177 \n",
"\n",
" median_auc_a median_auc_b mean_auc_diff ci_low ci_high cliffs_delta \\\n",
"0 155.035198 98.100125 31.68563 7.1042 58.820452 0.6 \n",
Expand All @@ -485,12 +485,12 @@
],
"source": [
"compare_auc(logger, \n",
" method_a= \"LLaMEA-SAGE\",\n",
" method_a= \"LLaMEA-CFG\",\n",
" method_b=\"LHNS:vns\",\n",
" budget=200,\n",
" metric=\"fitness\",\n",
" test=\"wilcoxon\",\n",
" replace_labels={'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-SAGE'})"
" replace_labels={'ES': 'LLaMEA', 'ES-guided': 'LLaMEA-CFG'})"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion visualize_3.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@
" if label == \"ES\":\n",
" new_labels.append(\"LLaMEA\")\n",
" elif label == \"ES-guided\":\n",
" new_labels.append(\"LLaMEA-SAGE\")\n",
" new_labels.append(\"LLaMEA-CFG\")\n",
" elif label == \"LHNS:vns\":\n",
" new_labels.append(\"LHNS\")\n",
" else:\n",
Expand Down
Loading