[Question] Clarification on gradient behavior for DI,NGD + CG combination in BOAT

[ysfchx]

Required prerequisites

• I have read the documentation https://boat.readthedocs.io/en/main/.
• I have searched the Issue Tracker to make sure this question has not already been answered.

Question

Environment

• OS: Windows 11
• Python: 3.10
• PyTorch: 2.5.1 + CUDA 12.1
• boat-torch: 1.0.2
• Other deps: matplotlib 3.10.9, higher 0.2.1, numpy 1.26.4

What I’m trying to do
I am running the BOAT Data Hyper-Cleaning experiments with different algorithm combinations.
Specifically, I am investigating the DI,NGD + CG combination for lower-level and upper-level optimization.

What I have already tried

• Ran data_hyper_cleaning.py --gm_op DI,NGD --na_op CG
• Generated loss_log.json and plotted convergence curves using plot_loss_curve.py

Observed behavior

• Lower-level loss (blue curve) initially rises slightly before decreasing
• Upper-level loss (orange curve) decreases steadily

Questions

• Is the initial rise in lower-level loss expected due to upper-level gradient influence?
• Are there recommended practices to stabilize lower-level convergence for this combination?

Extra context

• Relevant figures are attached in the report showing the convergence curves.

loss_curve_DI,NGD_CG.json