Add Pre-compilation Support for clails Applications

[tamurashingo]

Overview

When running a clails application in a production environment, there is a need to pre-compile the application code and generate an executable binary.

Currently, the quickstart guide and other documentation provide a method to load and run applications at runtime using the clails server command. However, for production environments, pre-compilation is preferable for the following reasons:

• Faster Startup: Binary execution is faster than interactive loading
• Resource Efficiency: Reduced memory usage and elimination of runtime compilation overhead
• Security: Source code is not deployed in the production environment
• Simplified Deployment: Easy containerization with Docker

Goals

Provide the following capabilities so that clails application developers can easily generate pre-compiled binaries for production environments:

1. Standardized Pre-compilation Process

   • Unified procedure assuming production environment
   • Standardized method for proper retrieval and configuration of environment variables

2. Build Tools/Scripts

   • Unified method to generate binaries from clails applications
   • Tool packaging in formats such as Makefile or shell scripts

3. Documentation

   • Detailed explanation of compilation procedure for production environments
   • Example of deployment method using Docker

Prerequisites

• When running clails applications in production, CLAILS_ENV=production is expected to be set
• Sensitive information such as database connection details is expected to be retrieved from environment variables
• SBCL (Steel Bank Common Lisp) is assumed to be used

Implementation Strategy

1. Application Preparation

Include the following in the template when generating clails applications:

• Definition of entry point function (run-app)
• Initialization processing (directory setup, environment variable configuration, database initialization)
• Retrieval of configuration values from environment variables

2. Build Process

Generate binaries in the following flow:

1. Install application dependencies (qlot install)
2. Load application (quickload)
3. Generate binary using save-lisp-and-die

3. Runtime Environment Variable Configuration

Required environment variables when running in production:

• CLAILS_ENV: Environment setting (fixed to production)
• CLAILS_PORT: Web server listening port (default: 5000)
• CLAILS_ADDRESS: Web server bind address (default: 127.0.0.1)
• Database connection information (project-specific environment variables)

Success Criteria

• Pre-compiled binary can be generated for the quickstart (TODO app)
• Generated binary runs correctly in production environment
• Can be executed in a Docker image
• Documentation is complete

Related PR/Issues

• Create documentation for production compilation procedure
• Implement Docker image generation

Creating Executable Binary

Creating Functions

Create app/main.lisp. Since save-lisp-and-die requires a function that takes no arguments, we need to define a wrapper function.

(in-package #:cl-user)
(defpackage #:todoapp/main
  (:nicknames #:todoapp)
  (:use #:cl
        #:clails)
  (:import-from #:clails/util
                #:env-or-default)
  (:import-from #:todoapp/config/database)
  (:export #:run-app))

(in-package #:todoapp/main)


(defun setup ()
  (push (uiop/os:getcwd) asdf:*central-registry*)
  (setf clails/environment:*project-dir* (uiop/os:getcwd))
  (setf clails/environment:*migration-base-dir* clails/environment:*project-dir*)

  (clails/environment:set-environment (env-or-default "CLAILS_ENV" "production"))

  (todoapp/config/database:initialize-database-config))


(defun run-app ()
  (setup)
  (let ((port (env-or-default "CLAILS_PORT" "5000"))
        (address (env-or-default "CLAILS_ADDRESS" "127.0.0.1")))
    (clails/cmd:server :port port :bind address)))

Prerequisites

Run the clails command to ensure that required modules are downloaded to CLAILS_CONF_DIR/.clails.

Creating the Executable File

Load the application from the directory where the clails application is located.

ros run

(push (uiop:getcwd) ql:*local-project-directories*)
(ql:quickload :todoapp)

Create the binary file.

(sb-ext:save-lisp-and-die
  #P"app-executable"
  :toplevel #'todoapp:run-app
  :executable t
  :compression t)

Running

Set environment variables as needed and run the application. Here we run with CLAILS_ENV = production.

Prepare the database for production beforehand.

CLAILS_ENV=production clails db:create
CLAILS_ENV=production clails db:migrate

Start the application.

CLAILS_DB_NAME=$PWD/tmp/todoapp-production.sqlite3 ./app-executable