treelite.sklearn.import_model ignores scikit-learn's learned missing-value direction for RandomForest/ExtraTrees/GradientBoosting

[cakedev0]

Description

When importing a RandomForestClassifier (or ExtraTrees*/GradientBoosting*) trained on data with missing values via treelite.sklearn.import_model, predictions on rows containing NaN are frequently wrong — sometimes by a large margin (>0.6 difference in predicted probability). Rows without any NaN predict correctly.

I traced this to treelite/sklearn/importer.py::import_model: for the RandomForest/ExtraTrees/GradientBoosting code path, only children_left, children_right, feature, threshold, and value are read off each scikit-learn tree_ and passed to the C API. The per-node tree_.missing_go_to_left array — which scikit-learn uses to decide which branch a NaN follows at each split (native missing-value support added to sklearn/tree/_tree.pyx) — is never read or forwarded. As a result, the imported treelite model falls back to its own default missing-value convention instead of the one the sklearn tree actually learned.

By contrast, HistGradientBoostingClassifier/Regressor import correctly: its importer passes the compiled nodes struct array wholesale, which already embeds a missing_go_to_left field per node, so missing-value routing is preserved. I verified this case matches predict_proba exactly (max abs diff 2.2e-16) with or without missing values.

Reproduction

import numpy as np
from sklearn.ensemble import RandomForestClassifier
import treelite.sklearn
import treelite.gtil

rng = np.random.RandomState(0)
n = 500
X = rng.randn(n, 5)
X[rng.rand(*X.shape) < 0.3] = np.nan  # ~30% missing values
y = (rng.rand(n) > 0.5).astype(int)

clf = RandomForestClassifier(n_estimators=50, random_state=0).fit(X, y)
sk_proba = clf.predict_proba(X)[:, 1]

tl_model = treelite.sklearn.import_model(clf)
tl_proba = np.array(treelite.gtil.predict(tl_model, X)).reshape(n, -1)[:, -1]

has_nan = np.isnan(X).any(axis=1)
diff = np.abs(sk_proba - tl_proba)

print(f"rows without NaN: {(diff[~has_nan] > 1e-6).sum()} / {(~has_nan).sum()} mismatched")
print(f"rows with NaN:    {(diff[has_nan] > 1e-6).sum()} / {has_nan.sum()} mismatched "
      f"(max abs diff {diff[has_nan].max():.3f})")

Output:

rows without NaN: 0 / 70 mismatched
rows with NaN:    426 / 430 mismatched (max abs diff 0.660)

Expected behavior

treelite.gtil.predict should match clf.predict_proba for all rows, including those with missing values, the same way it already does for HistGradientBoostingClassifier.

Suggested fix

In import_model, read tree.missing_go_to_left alongside children_left/children_right/etc. and forward it through to the corresponding TreeliteLoadSKLearn* C API calls (which would need a new parameter to accept it, mirroring how missing_go_to_left/default_left is handled elsewhere in treelite's tree builder API).

Environment

• treelite 4.7.1
• scikit-learn 1.10.dev0 (behavior applies to any sklearn version ≥1.3, when tree_.missing_go_to_left was introduced)
• Python 3.12.3

[chyunsu3]

Upcoming release of Treelite (4.8.0) will have support for missing values: #695.