Skip to content

Fix pairwise inference task index selection#505

Open
SergeiNikolenko wants to merge 3 commits into
OpenADMET:mainfrom
SergeiNikolenko:fix/inference-pairwise-variable-j
Open

Fix pairwise inference task index selection#505
SergeiNikolenko wants to merge 3 commits into
OpenADMET:mainfrom
SergeiNikolenko:fix/inference-pairwise-variable-j

Conversation

@SergeiNikolenko
Copy link
Copy Markdown
Contributor

@SergeiNikolenko SergeiNikolenko commented Mar 4, 2026

Description

Fix pairwise inference dataframe generation to use explicit task index selection (task_idx) and add regression tests for single-task and multitask pairwise paths.

Status

  • Ready to go

Developers certificate of origin

SergeiNikolenko and others added 3 commits February 26, 2026 22:52
The variable j in _generate_pairwise_df was the leaked loop variable
from the list comprehension [smiles[j] for i, j in pairs], not the
intended task index. This caused predictions to always use the wrong
column. Add an explicit task_idx parameter and pass it from the caller
where j is the proper enumerate index over tasknames.

Signed-off-by: Nikolenko.Sergei <Nikolenko.Sergei@icloud.com>
@SergeiNikolenko
Copy link
Copy Markdown
Contributor Author

Closing per author request during fork cleanup/reset.

@SergeiNikolenko SergeiNikolenko deleted the fix/inference-pairwise-variable-j branch March 4, 2026 07:56
@SergeiNikolenko SergeiNikolenko restored the fix/inference-pairwise-variable-j branch March 4, 2026 08:01
@hmacdope
Copy link
Copy Markdown
Contributor

@SergeiNikolenko does this have a specific task that is not working or issue that this is associated with? I wasn't aware that anything was wrong.

@SergeiNikolenko
Copy link
Copy Markdown
Contributor Author

Hi, yes, this is fixing a real bug in the pairwise inference path.

Before this change, _generate_pairwise_df was using predictions[:, j], where j came from the generated pair (i, j) rather than from the task index. That means the selected prediction column depended on the pair indices instead of the output task.

In practice, that could lead to two problems:

  1. for multitask pairwise inference, predictions could be written from the wrong output column;
  2. for single-task pairwise inference, this could hit an out-of-bounds column index depending on the generated pairs.

I added tests covering both the task index selection and the pairwise inference path.

Copy link
Copy Markdown
Contributor

@smcolby smcolby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix to inference.py is correct and the core test (test_generate_pairwise_df_uses_task_idx_column) is solid. A few changes are required before merge.

1. Test style: replace monkeypatch + hand-rolled dummies with pytest-mock

test_predict_single_task_pairwise_uses_column_zero and test_predict_pairwise_multitask_passes_task_idx both use monkeypatch.setattr with custom nested dummy classes (DummyPairwiseFeaturizer, DummyModel, fake_loader, fake_generate_pairwise_df). We're trying to use pytest-mock with mocker.patch(..., autospec=True) for this sort of thing, and will be updating the contribution guidelines soon.

Please rewrite both tests using the mocker fixture, e.g.:

def test_predict_single_task_pairwise_uses_column_zero(mocker, tmp_path):
    mock_load = mocker.patch(
        "openadmet.models.inference.inference.load_anvil_model_and_metadata",
        autospec=True,
    )
    mock_load.return_value = (...)
    ...

test_generate_pairwise_df_uses_task_idx_column is fine as-is — keep it.

2. DummyPairwiseFeaturizer is copy-pasted verbatim between the twomonkeypatch tests. Once rewritten with mocker, this duplication goes away naturally.

@dwwest dwwest self-requested a review May 19, 2026 19:08
Copy link
Copy Markdown
Contributor

@dwwest dwwest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thank you for catching this bug. Just a quick cosmetic change, plus Sean's requests re: testing, and it should be good to go.

predictions = model.predict(X_feat, accelerator=accelerator)
std = np.full(predictions.shape, np.nan)

for j, taskname in enumerate(tasknames):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change j to task_idx for consistency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants