Different results on Windows/Mac versus Linux

[koenderks]

When testing bain in JASP we came across the following inconsistencies in the results. Maybe you have an idea where this originates and how to tackle these differences.

Windows/Mac

data("sesamesim")
formula <- age ~ site + peabody + prenumb + postnumb + funumb - 1
hypothesis <- "site1 = site2 = site3 = site4 = site5;site1 < site2 < site3 < site4 < site5;site1 > site2 > site3 > site4 > site5"
fraction <- 1
standardized <- FALSE
sesamesim$site <- factor(sesamesim$site)
fit <- stats::lm(formula = formula, data = sesamesim)
bainResult <- bain::bain(x = fit, hypothesis = hypothesis, fraction = fraction, standardize = standardized)
> print(bainResult)

Bayesian informative hypothesis testing for an object of class lm (ANCOVA):
   Fit   Com   BF.u  BF.c  PMPa  PMPb 
H1 0.000 0.000 0.000 0.000 0.015 0.000
H2 0.000 0.007 0.007 0.007 0.985 0.007
H3 0.000 0.007 0.000 0.000 0.000 0.000
Hu                               0.993
Hypotheses:
  H1: site1=site2=site3=site4=site5
  H2: site1<site2<site3<site4<site5
  H3: site1>site2>site3>site4>site5
Note: BF.u denotes the Bayes factor of the hypothesis at hand versus the unconstrained hypothesis Hu. BF.c denotes the Bayes factor of the hypothesis at hand versus its complement.

>  bainResult$BFmatrix
             H1           H2         H3
H1  1.000000000 1.476255e-02   30175.92
H2 67.738958866 1.000000e+00 2044085.52
H3  0.000033139 4.892163e-07       1.00

Linux

data("sesamesim")
formula <- age ~ site + peabody + prenumb + postnumb + funumb - 1
hypothesis <- "site1 = site2 = site3 = site4 = site5;site1 < site2 < site3 < site4 < site5;site1 > site2 > site3 > site4 > site5"
fraction <- 1
standardized <- FALSE
sesamesim$site <- factor(sesamesim$site)
fit <- stats::lm(formula = formula, data = sesamesim)
bainResult <- bain::bain(x = fit, hypothesis = hypothesis, fraction = fraction, standardize = standardized)
> print(bainResult)

Bayesian informative hypothesis testing for an object of class lm (ANCOVA):
   Fit   Com   BF.u  BF.c  PMPa  PMPb 
H1 0.000 0.000 0.000 0.000 0.014 0.000
H2 0.000 0.007 0.007 0.007 0.986 0.007
H3 0.000 0.006 0.000 0.000 0.000 0.000
Hu                               0.992
Hypotheses:
  H1: site1=site2=site3=site4=site5
  H2: site1<site2<site3<site4<site5
  H3: site1>site2>site3>site4>site5
Note: BF.u denotes the Bayes factor of the hypothesis at hand versus the unconstrained hypothesis Hu. BF.c denotes the Bayes factor of the hypothesis at hand versus its complement.

> bainResult$BFmatrix
             H1           H2         H3
H1 1.000000e+00 1.404837e-02   22430.45
H2 7.118265e+01 1.000000e+00 1596659.07
H3 4.458225e-05 6.263078e-07       1.00

As you can see, the results between Mac/Windows and Linux diverge (for example BF23 = 30175.92 on windows versus BF23 = 22430.45 on Linux).

[cjvanlissa]

@koenderks I'm pretty sure this is due to the different implementation of the random number generator at the OS level. I don't think any action is required/possible. Have you any further insight?