Error when installing in docker image

[ff6347]

When I try to build the fhpredict (v.0.10.1) docker image (locally) I get the following error.

Installing package into '/usr/local/lib/R/site-library'
(as 'lib' is unspecified)
* installing *source* package 'kwb.flusshygiene' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading

 *** caught illegal operation ***
address 0x7fdb621a2e20, cause 'illegal operand'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)
 2: library.dynam(lib, package, package.lib)
 3: loadNamespace(name)
 4: doTryCatch(return(expr), name, parentenv, handler)
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6: tryCatchList(expr, classes, parentenv, handlers)
 7: tryCatch(loadNamespace(name), error = function(e) {    warning(gettextf("namespace %s is not available and has been replaced\nby .GlobalEnv when processing object %s",         sQuote(name)[1L], sQuote(where)), domain = NA, call. = FALSE,         immediate. = TRUE)    .GlobalEnv})
 8: ..getNamespace(c("rstan", "2.19.2"), "")
 9: get(classMetaName(cl), where)
10: methods::cacheMetaData(ns, TRUE, ns)
11: loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]])
12: namespaceImport(ns, loadNamespace(i, c(lib.loc, .libPaths()),     versionCheck = vI[[i]]), from = package)
13: loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]])
14: asNamespace(ns)
15: namespaceImportFrom(ns, loadNamespace(j <- i[[1L]], c(lib.loc,     .libPaths()), versionCheck = vI[[j]]), i[[2L]], from = package)
16: loadNamespace(package = package, lib.loc = lib.loc, keep.source = keep.source,     keep.parse.data = keep.parse.data, partial = TRUE)
17: withCallingHandlers(expr, packageStartupMessage = function(c) invokeRestart("muffleMessage"))
18: suppressPackageStartupMessages(loadNamespace(package = package,     lib.loc = lib.loc, keep.source = keep.source, keep.parse.data = keep.parse.data,     partial = TRUE))
19: code2LazyLoadDB(package, lib.loc = lib.loc, keep.source = keep.source,     keep.parse.data = keep.parse.data, compress = compress, set.install.dir = set.install.dir)
20: tools:::makeLazyLoading("kwb.flusshygiene", "/usr/local/lib/R/site-library/00LOCK-kwb.flusshygiene/00new",     keep.source = FALSE, keep.parse.data = FALSE, set.install.dir = "/usr/local/lib/R/site-library/kwb.flusshygiene")
An irrecoverable exception occurred. R is aborting now ...
Illegal instruction
ERROR: lazy loading failed for package 'kwb.flusshygiene'
* removing '/usr/local/lib/R/site-library/kwb.flusshygiene'
Error: Failed to install 'fhpredict' from GitHub:
  Failed to install 'kwb.flusshygiene' from GitHub:
  (converted from warning) installation of package '/tmp/RtmpJU9sr1/file64efa6e48/kwb.flusshygiene_0.3.0.tar.gz' had non-zero exit status
Execution halted
The command '/bin/sh -c R -e "remotes::install_github(\"kwb-r/fhpredict@v0.10.1\", build_vignettes = FALSE, force = TRUE, upgrade = \"never\")"' returned a non-zero code: 1

[ff6347]

I'm getting the same error (locally) with fhpredict v0.11.0

-  preparing 'kwb.flusshygiene':
v  checking DESCRIPTION meta-information ...
-  checking for LF line-endings in source and make files and shell scripts
-  checking for empty or unneeded directories
-  building 'kwb.flusshygiene_0.3.0.tar.gz'

Installing package into '/usr/local/lib/R/site-library'
(as 'lib' is unspecified)
* installing *source* package 'kwb.flusshygiene' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading

 *** caught illegal operation ***
address 0x7fc953f41e20, cause 'illegal operand'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)
 2: library.dynam(lib, package, package.lib)
 3: loadNamespace(name)
 4: doTryCatch(return(expr), name, parentenv, handler)
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6: tryCatchList(expr, classes, parentenv, handlers)
 7: tryCatch(loadNamespace(name), error = function(e) {    warning(gettextf("namespace %s is not available and has been replaced\nby .GlobalEnv when processing object %s",         sQuote(name)[1L], sQuote(where)), domain = NA, call. = FALSE,         immediate. = TRUE)    .GlobalEnv})
 8: ..getNamespace(c("rstan", "2.19.2"), "")
 9: get(classMetaName(cl), where)
10: methods::cacheMetaData(ns, TRUE, ns)
11: loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]])
12: namespaceImport(ns, loadNamespace(i, c(lib.loc, .libPaths()),     versionCheck = vI[[i]]), from = package)
13: loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]])
14: asNamespace(ns)
15: namespaceImportFrom(ns, loadNamespace(j <- i[[1L]], c(lib.loc,     .libPaths()), versionCheck = vI[[j]]), i[[2L]], from = package)
16: loadNamespace(package = package, lib.loc = lib.loc, keep.source = keep.source,     keep.parse.data = keep.parse.data, partial = TRUE)
17: withCallingHandlers(expr, packageStartupMessage = function(c) invokeRestart("muffleMessage"))
18: suppressPackageStartupMessages(loadNamespace(package = package,     lib.loc = lib.loc, keep.source = keep.source, keep.parse.data = keep.parse.data,     partial = TRUE))
19: code2LazyLoadDB(package, lib.loc = lib.loc, keep.source = keep.source,     keep.parse.data = keep.parse.data, compress = compress, set.install.dir = set.install.dir)
20: tools:::makeLazyLoading("kwb.flusshygiene", "/usr/local/lib/R/site-library/00LOCK-kwb.flusshygiene/00new",     keep.source = FALSE, keep.parse.data = FALSE, set.install.dir = "/usr/local/lib/R/site-library/kwb.flusshygiene")
An irrecoverable exception occurred. R is aborting now ...
Illegal instruction
ERROR: lazy loading failed for package 'kwb.flusshygiene'
* removing '/usr/local/lib/R/site-library/kwb.flusshygiene'
Error: Failed to install 'fhpredict' from GitHub:
  Failed to install 'kwb.flusshygiene' from GitHub:
  (converted from warning) installation of package '/tmp/RtmpQ1Jr3T/filea22f4b5a/kwb.flusshygiene_0.3.0.tar.gz' had non-zero exit status
Execution halted
The command '/bin/sh -c R -e "remotes::install_github(\"kwb-r/fhpredict@v0.11.0\", build_vignettes = FALSE, force = TRUE, upgrade = \"never\")"' returned a non-zero code: 1

[mrustl]

May be the R package "rstan" is not available anymore in version 2.19.2 (seems that the package was upgraded on CRAN to 2.19.3, https://cran.r-project.org/web/packages/rstan/index.html) ?

[ff6347]

The fhpredict image is based on our opencpu-base which builds fine. There we install the packages from the microsoft snapshot. See these lines.

https://github.com/technologiestiftung/flusshygiene/blob/18bfa05db6d417377cc170775158ab2c02cc0019/packages/opencpu-base/Dockerfile#L25-L26

How can we produce predictable builds?

[ff6347]

My base image has rstan v 2.19.2 installed. So this might not be the problem.

		Information on package 'rstan'

Description:

Package:              rstan
Encoding:             UTF-8
Type:                 Package
Title:                R Interface to Stan
Version:              2.19.2
Date:                 2019-07-08
Authors@R:            c(person("Jiqiang", "Guo", email =
                      "guojq28@gmail.com", role = "aut"),
                      person("Jonah", "Gabry", email =
                      "jsg2201@columbia.edu", role = "aut"),
                      person("Ben", "Goodrich", email =
                      "benjamin.goodrich@columbia.edu", role = c("cre",
                      "aut")), person("Daniel", "Lee", role = "ctb"),
                      person("Krzysztof", "Sakrejda", role = "ctb"),
                      person("Modrak", "Martin", role = "ctb"),
                      person("Trustees of", "Columbia University", role
                      = "cph"), person("Oleg", "Sklyar", role = "cph",
                      comment = "R/cxxfunplus.R"), person("The R",
                      "Core Team", role = "cph", comment = "R/pairs.R,
                      R/dynGet.R"), person("Jens",
                      "Oehlschlaegel-Akiyoshi", role = "cph", comment =
                      "R/pairs.R"))
Description:          User-facing R functions are provided to parse,
                      compile, test, estimate, and analyze Stan models
                      by accessing the header-only Stan library
                      provided by the 'StanHeaders' package. The Stan
                      project develops a probabilistic programming
                      language that implements full Bayesian
                      statistical inference via Markov Chain Monte
                      Carlo, rough Bayesian inference via 'variational'
                      approximation, and (optionally penalized) maximum
                      likelihood estimation via optimization. In all
                      three cases, automatic differentiation is used to
                      quickly and accurately evaluate gradients without
                      burdening the user with the need to derive the
                      partial derivatives.
License:              GPL (>= 3)
NeedsCompilation:     yes
Imports:              methods, stats4, inline, gridExtra (>= 2.0.0),
                      Rcpp (>= 0.12.0), loo (>= 2.0.0), pkgbuild
Depends:              R (>= 3.4.0), StanHeaders (> 2.18.1), ggplot2 (>=
                      2.0.0)
LinkingTo:            Rcpp (>= 0.12.0), RcppEigen (>= 0.3.3.3.0), BH
                      (>= 1.69.0), StanHeaders (> 2.18.1)
Suggests:             RUnit, RcppEigen (>= 0.3.3.3.0), BH (>= 1.66),
                      parallel, KernSmooth, shinystan (>= 2.3.0),
                      bayesplot (>= 1.5.0), rmarkdown, rstantools,
                      rstudioapi, Matrix, knitr (>= 1.15.1)
URL:                  http://discourse.mc-stan.org, http://mc-stan.org
BugReports:           https://github.com/stan-dev/rstan/issues/
VignetteBuilder:      knitr
SystemRequirements:   GNU make, pandoc
RoxygenNote:          5.0.1
Packaged:             2019-07-09 07:03:28 UTC; ben
Author:               Jiqiang Guo [aut], Jonah Gabry [aut], Ben
                      Goodrich [cre, aut], Daniel Lee [ctb], Krzysztof
                      Sakrejda [ctb], Modrak Martin [ctb], Trustees of
                      Columbia University [cph], Oleg Sklyar [cph]
                      (R/cxxfunplus.R), The R Core Team [cph]
                      (R/pairs.R, R/dynGet.R), Jens
                      Oehlschlaegel-Akiyoshi [cph] (R/pairs.R)
Maintainer:           Ben Goodrich <benjamin.goodrich@columbia.edu>
Repository:           CRAN
Date/Publication:     2019-07-09 12:40:03 UTC
Built:                R 3.6.2; x86_64-pc-linux-gnu; 2020-03-04 06:04:55
                      UTC; unix

Index:

As.mcmc.list            Create an mcmc.list from a stanfit object
Rhat                    Convergence and efficiency diagnostics for
                        Markov Chains
check_hmc_diagnostics   Check HMC diagnostics after sampling
expose_stan_functions   Expose user-defined Stan functions to R for
                        testing and simulation
extract                 Extract samples from a fitted Stan model
extract_sparse_parts    Extract the compressed representation of a
                        sparse matrix
gqs                     Draw samples of generated quantities from a
                        Stan model
is.array.stanfit        Create array, matrix, or data.frame objects
                        from samples in a 'stanfit' object
log_prob                'log_prob' and 'grad_log_prob' functions
loo.stanfit             Approximate leave-one-out cross-validation
lookup                  Look up the Stan function that corresponds to a
                        R function or name.
makeconf_path           Obtain the full path of file 'Makeconf'
monitor                 Compute summaries of MCMC draws and monitor
                        convergence
optimizing              Obtain a point estimate by maximizing the joint
                        posterior
pairs.stanfit           Create a matrix of output plots from a
                        'stanfit' object
plot,stanfit-method     Plots for stanfit objects
print.stanfit           Print a summary for a fitted model represented
                        by a 'stanfit' object
read_rdump              Read data in an R dump file to a list
read_stan_csv           Read CSV files of samples generated by (R)Stan
                        into a 'stanfit' object
rstan-package           RStan - the R interface to Stan
rstan-plotting-functions
                        RStan Plotting Functions
rstan.package.skeleton
                        Create a Skeleton for a New Source Package with
                        Stan Programs
rstan_gg_options        Set default appearance options
rstan_options           Set and read options used in RStan
sampling                Draw samples from a Stan model
sbc                     Simulation Based Calibration (sbc)
set_cppo                Defunct function to set the compiler
                        optimization level
sflist2stanfit          Merge a list of stanfit objects into one
stan                    Fit a model with Stan
stan_demo               Demonstrate examples included in Stan
stan_diag               RStan Diagnostic plots
stan_model              Construct a Stan model
stan_plot               ggplot2 for RStan
stan_rdump              Dump the data for a Stan model to R dump file
                        in the limited format that Stan can read.
stan_version            Obtain the version of Stan
stanc                   Translate Stan model specification to C++ code
stanfit-class           Class 'stanfit': fitted Stan model
stanmodel-class         Class representing model compiled from C++
summary,stanfit-method
                        Summary method for stanfit objects
traceplot               Markov chain traceplots
vb                      Run Stan's variational algorithm for
                        approximate posterior sampling

Further information is available in the following vignettes in
directory '/usr/local/lib/R/site-library/rstan/doc':

SBC: Simulation Based Calibration (source, pdf)
external: Interfacing with External C++ Code (source, pdf)
rstan: RStan (source, pdf)
stanfit-objects: stanfit objects (source, pdf)

[hsonne]

@fabianmoronzirfas FYI: I am currently trying to reproduce the error on my private Ubuntu laptop

Installing the "base" packages from MRAN snapshot...

• 1st try from within RStudio -> RStudio crashed
• 2nd try from within R running in console... laptop is smoking... and crashing!

virtual memory exhausted: Cannot allocate memory
make: *** [/usr/lib/R/etc/Makeconf:175: stan_files/jm.o] Error 1
rm stan_files/jm.cc stan_files/polr.cc
ERROR: compilation failed for package ‘rstanarm’
* removing ‘/home/hauke/tmp/r-lib/rstanarm’

Hm, on Windows the code similar to your docker files worked fine...

MRAN <- "https://mran.microsoft.com/snapshot/2019-12-18"

lib <- kwb.utils::createDirectory("~/tmp/r-lib")

withr::with_libpaths(lib, {
  install.packages(lib = lib, pkgs = c(
    'askpass' ,'assertthat' ,'aws.s3' ,'backports' ,'base64enc' ,'bayesplot' 
    ,'BH' ,'bitops' ,'callr' ,'caret' ,'checkmate' ,'classInt' ,'cli' ,
    'colorspace' ,'colourpicker' ,'crayon' ,'crosstalk' ,'curl' ,'DBI' ,'desc' ,
    'digest' ,'dplyr' ,'DT' ,'dygraphs' ,'e1071' ,'ellipsis' ,'fansi' ,'fs' ,
    'ggplot2' ,'ggridges' ,'glue' ,'gridExtra' ,'gtable' ,'gtools' ,'htmltools' ,
    'htmlwidgets' ,'httpuv' ,'httr' ,'igraph' ,'inline' ,'jsonlite' ,'labeling' ,
    'later' ,'lazyeval' ,'lme4' ,'lmtest' ,'loo' ,'lubridate' ,'magrittr' ,
    'markdown' ,'matrixStats' ,'mime' ,'miniUI' ,'minqa' ,'munsell' ,'nloptr' ,
    'openssl' ,'packrat' ,'pillar' ,'pkgbuild' ,'pkgconfig' ,'plogr' ,'plyr' ,
    'prettyunits' ,'processx' ,'promises' ,'ps' ,'purrr' ,'R6' ,'raster' ,
    'RColorBrewer' ,'Rcpp' ,'RcppEigen' ,'RCurl' ,'reshape2' ,'rlang' ,
    'rprojroot' ,'rsconnect' ,'rstan' ,'rstanarm' ,'rstantools' ,'rstudioapi' ,
    'scales' ,'sf' ,'shiny' ,'shinyjs' ,'shinystan' ,'shinythemes' ,
    'sourcetools' ,'sp' ,'StanHeaders' ,'stringi' ,'stringr' , 'svglite', 'sys' ,
    'threejs' ,'tibble' ,'tidyr' ,'tidyselect' ,'units' ,'utf8' ,'vctrs' ,
    'viridisLite' ,'withr' ,'xfun' ,'xml2' ,'xtable' ,'xts' ,'yaml' ,'zeallot' ,
    'zoo'), repo = MRAN)
})

withr::with_libpaths(lib, {
  remotes::install_github(
    "kwb-r/fhpredict@v0.11.0", build_vignettes = FALSE, force = TRUE, 
    upgrade = "never"
  )
})

[ff6347]

cool. But you don't have docker right?

[hsonne]

Right.
@fabianmoronzirfas Please consider my proposal here: KWB-R/fhpredict#61 (comment)