Python-based builds for AWS Lambda Layers#2
Python-based builds for AWS Lambda Layers#2zymergen-jringgenberg wants to merge 7 commits intoInformaticsMatters:masterfrom
Conversation
tdudgeon
left a comment
There was a problem hiding this comment.
Joe, thanks for this, and sorry it took a while to get round to review it.
My main concern here is that as this seems to be building with RDKit Release_2018_09_3 then these changes should go on the Release_2019_03* branches, probably start with the Release_2019_03 branch? I think this is better than committing on master.
Otherwise not much to say. In two of the files you are applying the patch for the cartridge build, but I don't think that's necessary as you're not building cartridge?
And I wonder longer term if building the lambda artifacts and images can just be incorporated into the standard build process, but think we need to understand the process better first.
|
Thanks for the comments! Good catch with the cartridge patching step - I've stripped that out of both files. And that makes sense about doing this on the release branches - whatever you prefer is fine. I hadn't had time yet to try fixing the Python 3.6 build to work with the March release, but I might have a chance later this week. Obviously the Python 2.7 build won't work with current and future releases. I'll put these changes together against the |
These new scripts use the current two-container strategy to build RDKit and assemble an AWS Lambda Layer as a deployable zip file. As the Lambda environment is CentOS-like, the method for building Boost from source is reused here as well.
Note: these builds currently work with the 2018_09 release of RDKit. Subsequent releases of RDKit do not support Python 2.7 in any case.
The Python 2.7 layer can be used in conjunction with the AWS-provided SciPy layer, while the Python 3.6 layer packages NumPy because of dependency conflicts once deployed to Lambda. There might be ways around these with a bit more work.