[test]: Evaluate pseudo-labeling and data imbalance strategies

[Dinghye]

We want to investigate two related but separate directions:

1. Whether pseudo-labeling can improve model performance by using the current best model to generate labels for unlabeled/test data.
2. Whether data imbalance is affecting model performance, and what strategies may help mitigate it.

These two directions should be evaluated independently first, instead of assuming pseudo-labeling is the solution to the data imbalance problem.

Motivation

Pseudo-labeling may increase the effective training data size and improve generalization. Separately, data imbalance may cause the model to underperform on rare classes, rare height ranges, or underrepresented regions. We want to understand both issues through controlled experiments.

Tasks

Part A: Pseudo-labeling

• Use the current best model to generate pseudo-labels for unlabeled/test data.
• Define a filtering strategy for pseudo-labels, such as:
  • confidence threshold
  • prediction quality check
  • manual inspection of selected samples
• Add pseudo-labeled data into the training pipeline.
• Train and compare against the current baseline.

Part B: Data Imbalance Analysis

• Analyze whether the training data is imbalanced across relevant dimensions, such as:
  • foreground/background ratio
  • class distribution
  • height value distribution
  • region/tile distribution (We already have some on [test]: Collect and verify core information from label bands  #1 )
• Identify which imbalance patterns may hurt IoU or RMSE.
• Propose and test simple mitigation strategies if time allows, such as:
  • weighted loss
  • oversampling underrepresented samples
  • targeted augmentation
  • balanced sampling

Deliverables

• Pseudo-label generation script or pipeline.
• Experiment results for baseline vs. pseudo-labeling.
• Data imbalance analysis summary.
  • Optional experiment results for imbalance mitigation strategies.
  • Recommendation on whether either direction should be continued.

Acceptance Criteria

• Pseudo-labeling is evaluated separately from imbalance handling.
• Validation IoU and RMSE are reported for pseudo-labeling experiments.
• Data imbalance is analyzed with quantitative evidence or clear visualizations.
• Any imbalance mitigation method is clearly separated from pseudo-labeling in the experiment table.

Notes

Please keep experiments reproducible:

• record config changes
• record checkpoint path
• report validation IoU and RMSE
• attach qualitative visualization if possible