diff --git a/README.org b/README.org index 3224041..b139a04 100644 --- a/README.org +++ b/README.org @@ -4,51 +4,59 @@ ** Prerequisites - You will need [[https://github.com/technomancy/leiningen][Leiningen]] 2.0.0 or above installed. + - [[https://adoptium.net/][Java]] 17 or newer (21 recommended; required by Ring / Jetty 12) + - [[https://github.com/technomancy/leiningen][Leiningen]] 2.11.0 or newer (2.12.x recommended) + - PostgreSQL available over TCP with a role/password your JDBC URL can use ** Development Mode *** Set Environment Variables & Secrets - Copy the example file, and update the values. + Copy the example file, then edit values to match your machine: #+BEGIN_SRC sh cp profiles_example.clj profiles.clj #+END_SRC + ~profiles.clj~ is gitignored. The example includes local defaults for: + + - ~DATABASE_URL~ — JDBC URL with user/password (not Unix peer auth) + - ~DATABASE_ENCRYPTION_KEY~ — key used to encrypt messages at rest + - ~SESSION_COOKIE_KEY~ — must be exactly 16 characters/bytes + - ~ENABLE_WORKERS~ — set ~false~ locally unless you want the expiry worker running + *** Setup Local Development Database - self-destruct uses postgresql; you'll need this available for local dev. to create a local dev database run the following commands. + Create a Postgres role and database that match ~profiles.clj~ (example uses + user/password ~selfdestruct~ / ~selfdestruct~): #+BEGIN_SRC sh - createdb self-destruct-dev - export DATABASE_URL=jdbc:postgresql://localhost:5432/self-destruct-dev?user=USER&password=PASSWORD + createuser selfdestruct + psql -c "ALTER USER selfdestruct WITH PASSWORD 'selfdestruct';" + createdb -O selfdestruct self-destruct-dev lein migrate #+END_SRC - JDBC connections typically need a username/password (peer auth via Unix sockets is not used). Update ~profiles.clj~ and ~DATABASE_URL~ to match your local Postgres role. + If you use a different role, update ~:database-url~ in ~profiles.clj~ (and + export ~DATABASE_URL~ if you override it in the shell). JDBC connections use + TCP auth; peer auth via Unix sockets is not used. -*** One alias to rule them all +*** Run the app - This alias should do all that is necessary in one step for local development, once migrations have been run. + After migrations have been applied: #+BEGIN_SRC sh lein dev #+END_SRC -*** Running the local development web server - - To start a web server for local live development of the application, run: - - #+BEGIN_SRC sh - lein ring server - #+END_SRC + That starts Garden CSS auto-compile and the Ring server (default port 8000). -*** Automatically Recompile CSS File on Changes + Equivalent pieces if you prefer to run them separately: #+BEGIN_SRC sh lein garden auto + lein ring server #+END_SRC *** Running Tests @@ -57,6 +65,11 @@ ./run-tests-local.sh #+END_SRC + This recreates ~self-destruct-test~, runs migrations, then ~lein test~. + Ensure the JDBC role in ~profiles.clj~ / ~DATABASE_URL~ can create that database + (or create it yourself and point ~DATABASE_URL~ at it). + + ** Continuous Integration Pull requests and pushes to ~master~ run on GitHub Actions (~.github/workflows/ci.yml~): @@ -65,28 +78,27 @@ ** Database Migrations - - the clojure migratus library is used to handle db schema changes. - - migrations are located in the ~resources/migrations~ folder. - - to create a new migration ~lein migratus create ~ and then edit the ~up~ and ~down~ files created. - - migrations are applied via ~lein migrate~ (local) or ~java -jar self-destruct.jar --migrate~ (production / Heroku release phase). - - for local development set ~DATABASE_URL~ to a JDBC URL Postgres will accept over TCP. + - Migratus handles schema changes; files live in ~resources/migrations~. + - Create a migration with ~lein migratus create ~, then edit the ~up~ / ~down~ SQL. + - Apply migrations with ~lein migrate~ (runs the app ~--migrate~ CLI) or + ~java -jar self-destruct.jar --migrate~ (production / Heroku release phase). ** Production Setup - you'll need to load the following environment variables in your production environment: + Required environment variables: - ~DATABASE_URL~ - ~DATABASE_ENCRYPTION_KEY~ - ~PORT~ - - ~SESSION_COOKIE_KEY~ + - ~SESSION_COOKIE_KEY~ (exactly 16 characters/bytes) - optionally you can customize the following variables in your production environment: + Optional: - - ~REPORTED_LOG_LEVEL~ (defaults to "warn"; sets min level to write to production log when using Sentry) - - ~LOG_APPENDER~ (options "println" "sentry") - - ~SENTRY_DSN~ (required only if you've set log-appender to "sentry") - - ~ENABLE_WORKERS~ (defaults to enabled; set to ~false~ to disable background workers) + - ~REPORTED_LOG_LEVEL~ (defaults to ~warn~; min level for Sentry when enabled) + - ~LOG_APPENDER~ (~println~ or ~sentry~) + - ~SENTRY_DSN~ (required only when ~LOG_APPENDER=sentry~) + - ~ENABLE_WORKERS~ (defaults to enabled; set ~false~ to disable background workers) - ~MESSAGE_EXPIRE_MINUTES~ (defaults to ~1440~ / 24 hours) - ~WORKER_DELAY_SECONDS~ (defaults to ~3600~ / 1 hour) diff --git a/project.clj b/project.clj index 372cb4d..0ec4ab6 100644 --- a/project.clj +++ b/project.clj @@ -1,40 +1,41 @@ (defproject self-destruct "0.1.0-SNAPSHOT" :description "run your own self destucting note service" :url "http://github.com/chadhs/self-destruct" - :min-lein-version "2.8.3" - :dependencies [[org.clojure/clojure "1.10.0"] + :min-lein-version "2.11.0" + :dependencies [[org.clojure/clojure "1.12.5"] ;;; core - [ring "1.7.1"] - [compojure "1.6.1"] + [ring/ring "1.15.5"] + [compojure "1.7.2"] ;;; environment - [environ "1.1.0"] - [org.clojure/tools.cli "0.4.2"] + [environ "1.2.0"] + [org.clojure/tools.cli "1.1.230"] ;;; database - [com.layerware/hugsql "0.4.9"] - [org.postgresql/postgresql "42.2.5"] - [migratus "1.2.3"] + [com.layerware/hugsql "0.5.3"] + [org.clojure/java.jdbc "0.7.12"] + [org.postgresql/postgresql "42.7.7"] + [migratus "1.6.7"] ;;; logging - [com.taoensso/timbre "4.10.0"] - [raven-clj "1.5.2"] ; timbre sentry support + [com.taoensso/timbre "6.8.0"] + [raven-clj "1.7.0"] ; timbre sentry support ;;; security - [buddy/buddy-core "1.6.0"] + [buddy/buddy-core "1.12.0-430"] ;;; ui - [hiccup "1.0.5"] - [garden "1.3.9"] + [hiccup/hiccup "2.0.0"] + [garden "1.3.10"] ;;; middleware - [ring/ring-defaults "0.3.2"] + [ring/ring-defaults "0.7.1"] ;;; data - [cheshire "5.8.1"] + [cheshire "5.13.0"] ;;; scheduling [tea-time "1.0.1"] ;;; hosted assests - [ring-webjars "0.2.0"] - [org.webjars/font-awesome "5.8.2"]] + [ring-webjars "0.3.1"] + [org.webjars/font-awesome "6.7.2"]] - :plugins [[lein-environ "1.1.0"] - [lein-ring "0.12.5"] - [migratus-lein "0.7.2"] + :plugins [[lein-environ "1.2.0"] + [lein-ring "0.12.6"] + [migratus-lein "0.7.3"] [lein-garden "0.3.0"] [lein-pdo "0.1.1"]] @@ -57,8 +58,8 @@ :migratus {:store :database :migration-dir "migrations" - :db {:connection-uri ~(or (System/getenv "DATABASE_URL") - "jdbc:postgresql://localhost:5432/self-destruct-dev?user=selfdestruct&password=selfdestruct")}} + :db {:jdbcUrl ~(or (System/getenv "DATABASE_URL") + "jdbc:postgresql://localhost:5432/self-destruct-dev?user=selfdestruct&password=selfdestruct")}} :profiles {:uberjar {:aot :all @@ -71,8 +72,8 @@ :profiles/test {} :profiles/prod {} :project/dev {:main self-destruct.core/-dev-main - :dependencies [[ring/ring-mock "0.4.0"]]} - :project/test {:dependencies[[ring/ring-mock "0.4.0"]]} + :dependencies [[ring/ring-mock "0.6.2"]]} + :project/test {:dependencies [[ring/ring-mock "0.6.2"]]} :project/prod {}} @@ -85,7 +86,9 @@ :prep-tasks ["clean" ["garden" "once"] "compile"] - :aliases {"migrate" ["migratus" "migrate"] + ;; Use the app CLI for migrations so we always run migratus 1.6+ / next.jdbc + ;; from project deps (migratus-lein still pins an older migratus). + :aliases {"migrate" ["run" "--" "--migrate"] "dev" ["pdo" ["garden" "auto"] ["ring" "server-headless"]]} diff --git a/src/self_destruct/config.clj b/src/self_destruct/config.clj index 52cd2f6..b71892c 100644 --- a/src/self_destruct/config.clj +++ b/src/self_destruct/config.clj @@ -1,8 +1,8 @@ (ns self-destruct.config - (:require [environ.core :as environ] - [migratus.core :as migratus] - [taoensso.timbre :as timbre] - [taoensso.timbre.appenders.3rd-party.sentry :as sentry])) + (:require [environ.core :as environ] + [migratus.core :as migratus] + [taoensso.timbre :as timbre]) + (:import [java.nio.charset StandardCharsets])) ;; database config @@ -17,7 +17,8 @@ (defn db-migration-config [] {:store :database :migration-dir "migrations" - :db (db-url)}) + ;; migratus 1.6+ requires a next.jdbc db spec map + :db {:jdbcUrl (db-url)}}) (defn run-db-migration [] ;; apply pending migrations @@ -38,8 +39,14 @@ ;; session cookie security config -(defn session-cookie-key [] - (environ/env :session-cookie-key)) +(defn session-cookie-key + "Return the cookie-store key as a 16-byte array (Ring's preferred form)." + [] + (let [key (environ/env :session-cookie-key)] + (cond + (bytes? key) key + (string? key) (.getBytes ^String key StandardCharsets/UTF_8) + :else key))) ;; logging config @@ -51,6 +58,14 @@ (or (environ/env :log-appender) "println")) +(defn- sentry-appender + "Load the Timbre community Sentry appender on demand so println-only + deployments do not need to initialize raven-clj at namespace load time." + [dsn] + (require 'taoensso.timbre.appenders.community.sentry) + ((resolve 'taoensso.timbre.appenders.community.sentry/sentry-appender) dsn)) + + (defn configure-logging [] (timbre/merge-config! {:appenders @@ -58,5 +73,5 @@ (= "println" (log-appender)) {:println {:output-fn :inherit}} (= "sentry" (log-appender)) {:sentry-appender (merge - (sentry/sentry-appender (environ/env :sentry-dsn)) + (sentry-appender (environ/env :sentry-dsn)) {:min-level (reported-log-level)})})})) diff --git a/src/self_destruct/core.clj b/src/self_destruct/core.clj index b58c053..772ca7f 100644 --- a/src/self_destruct/core.clj +++ b/src/self_destruct/core.clj @@ -55,31 +55,40 @@ ["-h" "--help"]]) +(defn- handle-cli-options + "Handle --help/--migrate. Returns an exit code when the process should + stop, or nil when the HTTP server should start." + [{:keys [options summary errors]}] + (cond + errors + (do + (timbre/error errors) + (timbre/info summary) + 1) + + (:help options) + (do + (timbre/info summary) + 0) + + (:migrate options) + (do + (timbre/info "running database migrations...") + (config/run-db-migration) + 0) + + :else + nil)) + + ;; main application entry point (defn -main [& args] - (let [{:keys [options arguments summary errors]} (parse-opts args cli-options) - port (:port options)] + (let [parsed (parse-opts args cli-options) + port (get-in parsed [:options :port])] ;; configure logging for all entrypoints (help/migrate/server) (config/configure-logging) - (cond - errors - (do - (timbre/error errors) - (timbre/info summary) - (System/exit 1)) - - (:help options) - (do - (timbre/info summary) - (System/exit 0)) - - (:migrate options) - (do - (timbre/info "running database migrations...") - (config/run-db-migration) - (System/exit 0)) - - :else + (if-let [exit-code (handle-cli-options parsed)] + (System/exit exit-code) (do (timbre/info "running init tasks") ;; workers only; logging already configured above @@ -91,11 +100,15 @@ ;; development mode main application entry point (defn -dev-main [& args] - (let [{:keys [options arguments summary errors]} (parse-opts args cli-options) - port (:port options)] - (do - (timbre/info "DEV: running init tasks") - (init) - (timbre/info (str "DEV: starting the app on port " port "...")) - (jetty/run-jetty (wrap-reload #'app) - {:port (Integer/valueOf port)})))) + (let [parsed (parse-opts args cli-options) + port (get-in parsed [:options :port])] + (config/configure-logging) + ;; honor --help/--migrate in dev as well (lein run defaults to -dev-main) + (if-let [exit-code (handle-cli-options parsed)] + (System/exit exit-code) + (do + (timbre/info "DEV: running init tasks") + (worker/launch-workers) + (timbre/info (str "DEV: starting the app on port " port "...")) + (jetty/run-jetty (wrap-reload #'app) + {:port (Integer/valueOf port)})))))