Maximize and report weighted metrics when regression weights are provided

[kmfrick]

Even when models are trained using regression weights, i.e. weights are passed to the function that does the actual training, the metrics computed, maximized/minimized, and reported are unweighted. This breaks optimality guarantees if I am not mistaken, e.g. for MSE you are taking the expectation over a population that your unweighted sample is not representative of.

postResample() used inline unweighted metric calculations and did not route through metric helpers, which were not weight-aware anyway.

defaultSummary() also did not consistently pass weights through to regression summaries.

Steps to reproduce

library(caret)
set.seed(54321)
n <- 120
x <- runif(n, -2, 2)
y <- 2 + 3*x + rnorm(n, sd=.4)
out <- sample(n, 20)
y[out] <- y[out] + rnorm(20, sd=8)     # large-noise points
w <- rep(1, n); w[out] <- 0.02         # downweight those points
df <- data.frame(x=x, y=y, w=w)

ctrl <- trainControl(method="cv", number=5, savePredictions="final")
fit <- train(x=df["x"], y=df$y, method="lm", weights=df$w, trControl=ctrl)
fit_lm <- lm(y ~ x, data=df, weights=w)

folds <- split(fit$pred, fit$pred$Resample)
rmse_u <- mean(vapply(folds, function(d) sqrt(mean((d$pred-d$obs)^2)), numeric(1)))
rmse_w <- mean(vapply(folds, function(d) sqrt(weighted.mean((d$pred-d$obs)^2, d$weights)), numeric(1)))
mae_u  <- mean(vapply(folds, function(d) mean(abs(d$pred-d$obs)), numeric(1)))
mae_w  <- mean(vapply(folds, function(d) weighted.mean(abs(d$pred-d$obs), d$weights), numeric(1)))

cat(sprintf("caret train RMSE: %.6f\n", fit$results$RMSE))
cat(sprintf("caret train MAE : %.6f\n", fit$results$MAE))
cat(sprintf("manual unweighted RMSE: %.6f\n", rmse_u))
cat(sprintf("manual weighted   RMSE: %.6f\n", rmse_w))
cat(sprintf("manual unweighted MAE : %.6f\n", mae_u))
cat(sprintf("manual weighted   MAE : %.6f\n", mae_w))
cat(sprintf("coef caret: %.6f %.6f\n", coef(fit$finalModel)[1], coef(fit$finalModel)[2]))
cat(sprintf("coef lm   : %.6f %.6f\n", coef(fit_lm)[1], coef(fit_lm)[2]))

Behaviur

Script outputs

caret train RMSE: 3.145790
caret train MAE : 1.351392
manual unweighted RMSE: 3.145790
manual weighted   RMSE: 0.632988
manual unweighted MAE : 1.351392
manual weighted   MAE : 0.328145
coef caret: 1.967979 3.006252
coef lm   : 1.967979 3.006252

Expected behavior

caret metric outputs should match weighted, not unweighted, manual ones.

Fix

• Added optional weights argument to postResample(), R2(), RMSE(), MAE()
• Updated defaultSummary() to pass a weights column (when present) into postResample().
• Updated roxygen docs

Tests

• Added test_that("weighted regression summaries are honored", ...) in pkg/caret/tests/testthat/test_misc.R.
• Added test_that("postResample argument matching remains backward compatible", ...) in pkg/caret/tests/testthat/test_misc.R.

Disclaimer

This PR was prepared with assistance from Codex, but I have experience with R so I can conduct more thorough testing if you want.