Add jobrequest technical docs

package.json

@@ -5,6 +5,6 @@
   "author": "Government Digital Service",
   "license": "MIT",
   "dependencies": {
-    "mermaid": "^11.16.1"
+    "mermaid": "^11.17.2"
   }
 }

source/kubernetes/job-requests/architecture/index.html.md

@@ -1,9 +1,278 @@
 ---
-title: Job Request Architecture
+title: JobRequest Architecture
 weight: 10
 layout: multipage_layout
 ---
 
-# Job Request Architecture
+# JobRequest Architecture
 
-Coming soon...
+## System Components
+
+* [govuk-job-request-operator](/repos/govuk-job-request-operator.html):
+  This includes:
+  * [Custom Resource Defitinitions
+    (CRDs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
+    for JobRequest (short name `jr`) and JobRequestReview (short name `jrr`)
+    resources
+  * [Kubernetes
+    Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)
+    which is responsible for reconcilling the JobRequest and JobRequestReview
+    resources
+  * [Kubernetes Mutating Admission
+  Policy](https://kubernetes.io/docs/reference/access-authn-authz/mutating-admission-policy/)
+    which is responsible for validating, and adding the AWS ARN of the
+    requesting user into annotations on the JobRequest and JobRequestReview
+    during admission
+  * [Garbage
+    collection](https://kubernetes.io/docs/concepts/architecture/garbage-collection/)
+    to clean up JobRequest and JobRequestReview resoureces more than 30 days old
+* [OPA
+  Gatekeeper](https://kubernetes.io/blog/2019/08/06/opa-gatekeeper-policy-and-governance-for-kubernetes/)
+  [policies](https://github.com/alphagov/govuk-infrastructure/tree/main/terraform/deployments/cluster-services/modules/gatekeeper)
+  responsible for validating API requests to create/update JobRequest and
+  JobRequestReview resources, and providing useful error messages to the end user
+  if the request is denied.
+* [govuk-cli](/repos/govuk-job-request-operator.html): A command line interface
+  with a jobrequest subcommand which is the primary interface developers are
+  expected to use to create and review JobRequests. This also does validation
+  prior to making create requests to the Kubernetes API to give the most friendly
+  error messages possible.
+
+## Expected usage
+
+The following describes the intended happy path from creation of a JobRequest to Job completion.
+
+1. User A uses `govuk-cli` to create a JobRequest, it prints out the CLI command which can be used to review the JobRequest, and waits to show the logs from the Job that will be created later.
+2. User A sends User B the printed out CLI command to run in order to review the request.
+3. User B runs the CLI command, reads and reviews the command intended to run, chooses to approve or reject it, then `govuk-cli` creates a JobRequestReview.
+4. Review outcome:
+    a. If user B rejects the JobRequest, assuming user A is still following the logs, user A sees a message telling them the request was rejected. This flow ends here.
+    b. If user B approves the JobRequest then continue this process
+5. The JobRequest operator changes the Status of the JobRequest to approved.
+6. The JobRequest operator reads the pod spec it needs to create from the target resource specified in the JobRequest
+7. The JobRequest operator creates a Kubernetes Job from the pod spec retrieved in the previous step, overriding the command to be the one approved in the JobRequest
+8. The JobRequest operator updates the Status of the JobRequest to include the name of the Job that was created
+9. The govuk-cli command that was run in step 1. sees the Job has been created, informs user A, and starts printing out the logs of the Job as they are produced.
+9. The JobRequest operator watches the status of the Job and updates the JobRequest status to include the current state of the Job.
+10. When the status of the Job reaches a terminal state, the govuk-cli stops following the logs and informs User A the Job has completed.
+
+### Sequence Diagram for an Approved JobRequest
+
+<pre lang="mermaid">
+
+<code>
+sequenceDiagram
+    actor Requester
+    actor Reviewer
+    participant k8sAPI as Kubernetes API
+    participant Operator
+
+    Requester->>k8sAPI: `govuk-cli create jobrequest... --follow`
+    k8sAPI->>Operator: Job Request Created
+    Requester->>Reviewer: Please review my request
+    Reviewer->>k8sAPI: `govuk-cli jobrequest review...` Approved
+    k8sAPI->>Operator: JobRequestReview Created
+    Operator->>k8sAPI: Create Job
+    k8sAPI->>Requester: Logs
+    k8sAPI->>Requester: More Logs
+    k8sAPI->>Requester: Job Complete
+</code>
+</pre>
+
+### Sequence Diagram for a Rejected JobRequest
+
+<pre lang="mermaid">
+
+<code>
+sequenceDiagram
+    actor Requester
+    actor Reviewer
+    participant k8sAPI as Kubernetes API
+    participant Operator
+
+    Requester->>k8sAPI: `govuk-cli create jobrequest... --follow`
+    k8sAPI->>Operator: JobRequest Created
+    Requester->>Reviewer: Please review my request
+    Reviewer->>k8sAPI: `govuk-cli jobrequest review...` Rejected
+    k8sAPI->>Operator: Rejected JobRequestReview Created
+    Operator->>k8sAPI: Update JobRequest to Rejected
+    k8sAPI->>Requester: JobRequest Rejected
+</code>
+</pre>
+
+## Garbage Collection
+
+Any time a JobRequest or JobRequestReview resource is presented for
+reconcilliation, if it has lived longer than the TTL duration (which is set
+for 720 hours (30 days)), it will be deleted.
+
+The [Kubernetes Controller Runtime
+configuration](https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/cache#Config)
+includes a SyncPeriod, any resources managed by the controller runtime will be
+presented to the operator every time the `SyncPeriod` has elapsed.
+
+The SyncPeriod in the [is configurable in the
+govuk-job-request-operator](https://github.com/alphagov/govuk-job-request-operator#configuration)
+by setting the `--resource-ttl` flag.
+
+## Architecture Diagrams
+
+### JobRequest state diagram
+
+<pre lang="mermaid">
+
+<code>
+stateDiagram
+    state "''" as NoState
+
+    [*] --> NoState
+    NoState --> Malformed
+    NoState --> Pending
+    Pending --> Rejected
+    Pending --> Approved
+    Approved --> Started
+    Approved --> Malformed
+    Started --> Complete
+    Started --> Failed
+    Started --> Malformed
+    Malformed --> [*]
+    Complete --> [*]
+    Failed --> [*]
+</code>
+</pre>
+
+### JobRequestReview state diagram
+
+<pre lang="mermaid">
+
+<code>
+stateDiagram
+    [*] --> Approved
+    [*] --> Rejected
+
+    Approved --> JobRequestNotFound
+    Approved --> JobRequestMalformed
+    Approved --> Conflict
+
+    Rejected --> JobRequestNotFound
+    Rejected --> JobRequestMalformed
+    Rejected --> Conflict
+
+    Approved --> [*]
+    Rejected --> [*]
+    JobRequestNotFound --> [*]
+    JobRequestMalformed --> [*]
+    Conflict --> [*]
+</code>
+</pre>
+
+### Creating a JobRequest Flowchart
+
+<pre lang="mermaid">
+
+<code>
+flowchart TD
+    RequesterA@{ shape: person }
+    RequesterB@{ shape: person }
+    govuk-cli
+
+    RequesterA--govuk-cli jobrequest create ...-->govuk-cli
+    govuk-cli--Create JobRequest-->k8sAPI
+
+    RequesterB--Create JobRequest-->k8sAPI
+
+    subgraph k8s[Kubernetes]
+        direction TD
+
+        k8sAPI[Kubernetes API]
+        MutatingAdmissionPolicy
+        gatekeeper[OPA Gatekeeper]
+        govuk-job-request-operator[Operator JobRequest controller]
+        etcd[(Etcd)]
+
+        k8sAPI-- Create JobRequest -->MutatingAdmissionPolicy
+        MutatingAdmissionPolicy--Create JobRequest-->gatekeeper
+        gatekeeper--Create JobRequest-->etcd
+
+        etcd--Created JobRequest-->govuk-job-request-operator
+
+        govuk-job-request-operator-->validateJobRequest{Validate}
+        validateJobRequest--valid\n\nSet JobRequest Pending-->k8sAPI
+        validateJobRequest--invalid\n\nSet JobRequest Malformed-->k8sAPI
+    end
+</code>
+</pre>
+
+### Reviewing a JobRequest Flowchart
+
+<pre lang="mermaid">
+
+<code>
+flowchart TD
+    RequesterA@{ shape: person }
+    RequesterB@{ shape: person }
+    govuk-cli
+
+    RequesterA--govuk-cli jobrequest review ...-->govuk-cli
+    govuk-cli--Create JobRequestReview-->k8sAPI
+
+    RequesterB--Create JobRequestReview-->k8sAPI
+
+    subgraph k8s[Kubernetes]
+        direction TD
+
+        k8sAPI[Kubernetes API]
+        MutatingAdmissionPolicy
+        gatekeeper[OPA Gatekeeper]
+        govuk-job-request-operator[Operator JobRequestReview controller]
+        etcd[(Etcd)]
+
+        k8sAPI-- Create JobRequestReview -->MutatingAdmissionPolicy
+        MutatingAdmissionPolicy-- Create JobRequestReview -->gatekeeper
+        gatekeeper-- Create JobRequestReview -->etcd
+
+        etcd-- Created JobRequestReview -->govuk-job-request-operator
+
+        govuk-job-request-operator-->validateJobRequestReview{Validate}
+        validateJobRequestReview-- invalid\n\nSet JobRequestReview Malformed -->k8sAPI
+
+        validateJobRequestReview-- valid -->jobRequestFound{JobRequest Exists?}
+        jobRequestFound-- found\n\nSet state of JobRequestReview to Approved/Rejected\n\nSet state of JobRequest to Approved/Rejected -->k8sAPI
+        jobRequestFound-- not-found\n\nSet state of JobRequestReview toJobRequestNotFound -->k8sAPI
+    end
+</code>
+</pre>
+
+### After a JobRequest has been Approved Flowchart
+
+<pre lang="mermaid">
+
+<code>
+flowchart TD
+    subgraph k8s[Kubernetes]
+        direction TD
+
+        k8sAPI[Kubernetes API]
+        MutatingAdmissionPolicy
+        gatekeeper[OPA Gatekeeper]
+        govuk-job-request-operator[Operator JobRequest controller]
+        etcd[(Etcd)]
+
+        etcd-- 1. JobRequest Updated -->govuk-job-request-operator
+        govuk-job-request-operator<-- 2. Get Pod/Deployment -->k8sAPI
+
+        govuk-job-request-operator-- 3. Create Job -->k8sAPI
+        k8sAPI-- 3. Create Job -->etcd
+
+        govuk-job-request-operator-- 4. Watch Job -->k8sAPI
+
+        etcd-- 5. Job State Change --> govuk-job-request-operator
+        govuk-job-request-operator-- 5. Update JobRequest with Job State -->k8sAPI
+        k8sAPI-- 5. Update JobRequest -->MutatingAdmissionPolicy
+        MutatingAdmissionPolicy-- 5. Update JobRequest -->gatekeeper
+        gatekeeper-- 5. Update JobRequest -->etcd
+
+    end
+
+</code>
+</pre>

source/kubernetes/job-requests/index.html.md

@@ -9,10 +9,33 @@ layout: multipage_layout
 If you are looking for how to create or review JobRequests, see the
 [JobRequests User Guide](/kubernetes/job-requests/user-guide/)
 
+If you are looking to understand how the JobRequest system works, see [the
+JobRequest architecture guide](/kubernetes/job-requests/architecture).
+
 ## What is the JobRequest system
 
-Coming soon...
+The JobRequest system allows for a user to run commands within containers in
+Kubernetes in a way which is:
+
+* Peer reviewed
+* Audited
+* Has isolated logs not mixed with regular application logs
+* Easy to use
+* Easy to see the logs for an executed command
+* Does not require the user to create and understand kubernetes manifests
+* Is garbage collected after a month
+
+## What is the process for executing a Job using JobRequests
+
+1. A user creates a JobRequest to run a command in Kubernetes, which will be
+   run in a Pod that is a copy of an existing Pod or Deployment.
+2. A different user reviews that request (either approving or rejecting).
+3. If approved a Kubernetes Job will be created which:
+    * Has its Pod spec copied from the requested Pod/Deployment
+    * Has the command of the Pod overridden with the command the user requested
 
 ## Why might you use it
 
-Coming soon...
+* To execute a Rake task in a Kubernetes cluster
+* To receive a peer review for a command you are going to run in a Kubernetes cluster
+* To have logs produced so you can see what your command did