Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 40 additions & 28 deletions README.org
Original file line number Diff line number Diff line change
Expand Up @@ -4,51 +4,59 @@

** Prerequisites

You will need [[https://github.com/technomancy/leiningen][Leiningen]] 2.0.0 or above installed.
- [[https://adoptium.net/][Java]] 17 or newer (21 recommended; required by Ring / Jetty 12)
- [[https://github.com/technomancy/leiningen][Leiningen]] 2.11.0 or newer (2.12.x recommended)
- PostgreSQL available over TCP with a role/password your JDBC URL can use


** Development Mode

*** Set Environment Variables & Secrets

Copy the example file, and update the values.
Copy the example file, then edit values to match your machine:

#+BEGIN_SRC sh
cp profiles_example.clj profiles.clj
#+END_SRC

~profiles.clj~ is gitignored. The example includes local defaults for:

- ~DATABASE_URL~ — JDBC URL with user/password (not Unix peer auth)
- ~DATABASE_ENCRYPTION_KEY~ — key used to encrypt messages at rest
- ~SESSION_COOKIE_KEY~ — must be exactly 16 characters/bytes
- ~ENABLE_WORKERS~ — set ~false~ locally unless you want the expiry worker running

*** Setup Local Development Database

self-destruct uses postgresql; you'll need this available for local dev. to create a local dev database run the following commands.
Create a Postgres role and database that match ~profiles.clj~ (example uses
user/password ~selfdestruct~ / ~selfdestruct~):

#+BEGIN_SRC sh
createdb self-destruct-dev
export DATABASE_URL=jdbc:postgresql://localhost:5432/self-destruct-dev?user=USER&password=PASSWORD
createuser selfdestruct
psql -c "ALTER USER selfdestruct WITH PASSWORD 'selfdestruct';"
createdb -O selfdestruct self-destruct-dev
lein migrate
#+END_SRC

JDBC connections typically need a username/password (peer auth via Unix sockets is not used). Update ~profiles.clj~ and ~DATABASE_URL~ to match your local Postgres role.
If you use a different role, update ~:database-url~ in ~profiles.clj~ (and
export ~DATABASE_URL~ if you override it in the shell). JDBC connections use
TCP auth; peer auth via Unix sockets is not used.

*** One alias to rule them all
*** Run the app

This alias should do all that is necessary in one step for local development, once migrations have been run.
After migrations have been applied:

#+BEGIN_SRC sh
lein dev
#+END_SRC

*** Running the local development web server

To start a web server for local live development of the application, run:

#+BEGIN_SRC sh
lein ring server
#+END_SRC
That starts Garden CSS auto-compile and the Ring server (default port 8000).

*** Automatically Recompile CSS File on Changes
Equivalent pieces if you prefer to run them separately:

#+BEGIN_SRC sh
lein garden auto
lein ring server
#+END_SRC

*** Running Tests
Expand All @@ -57,6 +65,11 @@
./run-tests-local.sh
#+END_SRC

This recreates ~self-destruct-test~, runs migrations, then ~lein test~.
Ensure the JDBC role in ~profiles.clj~ / ~DATABASE_URL~ can create that database
(or create it yourself and point ~DATABASE_URL~ at it).


** Continuous Integration

Pull requests and pushes to ~master~ run on GitHub Actions (~.github/workflows/ci.yml~):
Expand All @@ -65,28 +78,27 @@

** Database Migrations

- the clojure migratus library is used to handle db schema changes.
- migrations are located in the ~resources/migrations~ folder.
- to create a new migration ~lein migratus create <descriptive-name>~ and then edit the ~up~ and ~down~ files created.
- migrations are applied via ~lein migrate~ (local) or ~java -jar self-destruct.jar --migrate~ (production / Heroku release phase).
- for local development set ~DATABASE_URL~ to a JDBC URL Postgres will accept over TCP.
- Migratus handles schema changes; files live in ~resources/migrations~.
- Create a migration with ~lein migratus create <descriptive-name>~, then edit the ~up~ / ~down~ SQL.
- Apply migrations with ~lein migrate~ (runs the app ~--migrate~ CLI) or
~java -jar self-destruct.jar --migrate~ (production / Heroku release phase).


** Production Setup

you'll need to load the following environment variables in your production environment:
Required environment variables:

- ~DATABASE_URL~
- ~DATABASE_ENCRYPTION_KEY~
- ~PORT~
- ~SESSION_COOKIE_KEY~
- ~SESSION_COOKIE_KEY~ (exactly 16 characters/bytes)

optionally you can customize the following variables in your production environment:
Optional:

- ~REPORTED_LOG_LEVEL~ (defaults to "warn"; sets min level to write to production log when using Sentry)
- ~LOG_APPENDER~ (options "println" "sentry")
- ~SENTRY_DSN~ (required only if you've set log-appender to "sentry")
- ~ENABLE_WORKERS~ (defaults to enabled; set to ~false~ to disable background workers)
- ~REPORTED_LOG_LEVEL~ (defaults to ~warn~; min level for Sentry when enabled)
- ~LOG_APPENDER~ (~println~ or ~sentry~)
- ~SENTRY_DSN~ (required only when ~LOG_APPENDER=sentry~)
- ~ENABLE_WORKERS~ (defaults to enabled; set ~false~ to disable background workers)
- ~MESSAGE_EXPIRE_MINUTES~ (defaults to ~1440~ / 24 hours)
- ~WORKER_DELAY_SECONDS~ (defaults to ~3600~ / 1 hour)

Expand Down
55 changes: 29 additions & 26 deletions project.clj
Original file line number Diff line number Diff line change
@@ -1,40 +1,41 @@
(defproject self-destruct "0.1.0-SNAPSHOT"
:description "run your own self destucting note service"
:url "http://github.com/chadhs/self-destruct"
:min-lein-version "2.8.3"
:dependencies [[org.clojure/clojure "1.10.0"]
:min-lein-version "2.11.0"
:dependencies [[org.clojure/clojure "1.12.5"]
;;; core
[ring "1.7.1"]
[compojure "1.6.1"]
[ring/ring "1.15.5"]
[compojure "1.7.2"]
;;; environment
[environ "1.1.0"]
[org.clojure/tools.cli "0.4.2"]
[environ "1.2.0"]
[org.clojure/tools.cli "1.1.230"]
;;; database
[com.layerware/hugsql "0.4.9"]
[org.postgresql/postgresql "42.2.5"]
[migratus "1.2.3"]
[com.layerware/hugsql "0.5.3"]
[org.clojure/java.jdbc "0.7.12"]
[org.postgresql/postgresql "42.7.7"]
[migratus "1.6.7"]
;;; logging
[com.taoensso/timbre "4.10.0"]
[raven-clj "1.5.2"] ; timbre sentry support
[com.taoensso/timbre "6.8.0"]
[raven-clj "1.7.0"] ; timbre sentry support
;;; security
[buddy/buddy-core "1.6.0"]
[buddy/buddy-core "1.12.0-430"]
;;; ui
[hiccup "1.0.5"]
[garden "1.3.9"]
[hiccup/hiccup "2.0.0"]
[garden "1.3.10"]
;;; middleware
[ring/ring-defaults "0.3.2"]
[ring/ring-defaults "0.7.1"]
;;; data
[cheshire "5.8.1"]
[cheshire "5.13.0"]
;;; scheduling
[tea-time "1.0.1"]
;;; hosted assests
[ring-webjars "0.2.0"]
[org.webjars/font-awesome "5.8.2"]]
[ring-webjars "0.3.1"]
[org.webjars/font-awesome "6.7.2"]]


:plugins [[lein-environ "1.1.0"]
[lein-ring "0.12.5"]
[migratus-lein "0.7.2"]
:plugins [[lein-environ "1.2.0"]
[lein-ring "0.12.6"]
[migratus-lein "0.7.3"]
[lein-garden "0.3.0"]
[lein-pdo "0.1.1"]]

Expand All @@ -57,8 +58,8 @@

:migratus {:store :database
:migration-dir "migrations"
:db {:connection-uri ~(or (System/getenv "DATABASE_URL")
"jdbc:postgresql://localhost:5432/self-destruct-dev?user=selfdestruct&password=selfdestruct")}}
:db {:jdbcUrl ~(or (System/getenv "DATABASE_URL")
"jdbc:postgresql://localhost:5432/self-destruct-dev?user=selfdestruct&password=selfdestruct")}}


:profiles {:uberjar {:aot :all
Expand All @@ -71,8 +72,8 @@
:profiles/test {}
:profiles/prod {}
:project/dev {:main self-destruct.core/-dev-main
:dependencies [[ring/ring-mock "0.4.0"]]}
:project/test {:dependencies[[ring/ring-mock "0.4.0"]]}
:dependencies [[ring/ring-mock "0.6.2"]]}
:project/test {:dependencies [[ring/ring-mock "0.6.2"]]}
:project/prod {}}


Expand All @@ -85,7 +86,9 @@
:prep-tasks ["clean" ["garden" "once"] "compile"]


:aliases {"migrate" ["migratus" "migrate"]
;; Use the app CLI for migrations so we always run migratus 1.6+ / next.jdbc
;; from project deps (migratus-lein still pins an older migratus).
:aliases {"migrate" ["run" "--" "--migrate"]
"dev" ["pdo" ["garden" "auto"] ["ring" "server-headless"]]}


Expand Down
31 changes: 23 additions & 8 deletions src/self_destruct/config.clj
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
(ns self-destruct.config
(:require [environ.core :as environ]
[migratus.core :as migratus]
[taoensso.timbre :as timbre]
[taoensso.timbre.appenders.3rd-party.sentry :as sentry]))
(:require [environ.core :as environ]
[migratus.core :as migratus]
[taoensso.timbre :as timbre])
(:import [java.nio.charset StandardCharsets]))


;; database config
Expand All @@ -17,7 +17,8 @@
(defn db-migration-config []
{:store :database
:migration-dir "migrations"
:db (db-url)})
;; migratus 1.6+ requires a next.jdbc db spec map
:db {:jdbcUrl (db-url)}})

(defn run-db-migration []
;; apply pending migrations
Expand All @@ -38,8 +39,14 @@


;; session cookie security config
(defn session-cookie-key []
(environ/env :session-cookie-key))
(defn session-cookie-key
"Return the cookie-store key as a 16-byte array (Ring's preferred form)."
[]
(let [key (environ/env :session-cookie-key)]
(cond
(bytes? key) key
(string? key) (.getBytes ^String key StandardCharsets/UTF_8)
:else key)))


;; logging config
Expand All @@ -51,12 +58,20 @@
(or (environ/env :log-appender) "println"))


(defn- sentry-appender
"Load the Timbre community Sentry appender on demand so println-only
deployments do not need to initialize raven-clj at namespace load time."
[dsn]
(require 'taoensso.timbre.appenders.community.sentry)
((resolve 'taoensso.timbre.appenders.community.sentry/sentry-appender) dsn))


(defn configure-logging []
(timbre/merge-config!
{:appenders
(cond
(= "println" (log-appender)) {:println {:output-fn :inherit}}
(= "sentry" (log-appender)) {:sentry-appender
(merge
(sentry/sentry-appender (environ/env :sentry-dsn))
(sentry-appender (environ/env :sentry-dsn))
{:min-level (reported-log-level)})})}))
71 changes: 42 additions & 29 deletions src/self_destruct/core.clj
Original file line number Diff line number Diff line change
Expand Up @@ -55,31 +55,40 @@
["-h" "--help"]])


(defn- handle-cli-options
"Handle --help/--migrate. Returns an exit code when the process should
stop, or nil when the HTTP server should start."
[{:keys [options summary errors]}]
(cond
errors
(do
(timbre/error errors)
(timbre/info summary)
1)

(:help options)
(do
(timbre/info summary)
0)

(:migrate options)
(do
(timbre/info "running database migrations...")
(config/run-db-migration)
0)

:else
nil))


;; main application entry point
(defn -main [& args]
(let [{:keys [options arguments summary errors]} (parse-opts args cli-options)
port (:port options)]
(let [parsed (parse-opts args cli-options)
port (get-in parsed [:options :port])]
;; configure logging for all entrypoints (help/migrate/server)
(config/configure-logging)
(cond
errors
(do
(timbre/error errors)
(timbre/info summary)
(System/exit 1))

(:help options)
(do
(timbre/info summary)
(System/exit 0))

(:migrate options)
(do
(timbre/info "running database migrations...")
(config/run-db-migration)
(System/exit 0))

:else
(if-let [exit-code (handle-cli-options parsed)]
(System/exit exit-code)
(do
(timbre/info "running init tasks")
;; workers only; logging already configured above
Expand All @@ -91,11 +100,15 @@

;; development mode main application entry point
(defn -dev-main [& args]
(let [{:keys [options arguments summary errors]} (parse-opts args cli-options)
port (:port options)]
(do
(timbre/info "DEV: running init tasks")
(init)
(timbre/info (str "DEV: starting the app on port " port "..."))
(jetty/run-jetty (wrap-reload #'app)
{:port (Integer/valueOf port)}))))
(let [parsed (parse-opts args cli-options)
port (get-in parsed [:options :port])]
(config/configure-logging)
;; honor --help/--migrate in dev as well (lein run defaults to -dev-main)
(if-let [exit-code (handle-cli-options parsed)]
(System/exit exit-code)
(do
(timbre/info "DEV: running init tasks")
(worker/launch-workers)
(timbre/info (str "DEV: starting the app on port " port "..."))
(jetty/run-jetty (wrap-reload #'app)
{:port (Integer/valueOf port)})))))
Loading