Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 22 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ jobs:
# the minimum line-coverage threshold.
# `make test-cover` enables -race via RACE=on (the Makefile default),
# writes coverage.out, and exits non-zero if coverage < COVERAGE_MIN.
# COVERAGE_MIN defaults to 60 in the Makefile — that is the single
# COVERAGE_MIN defaults to 70 in the Makefile — that is the single
# source of truth; raise it there as coverage grows.
- name: Test (race detector + coverage gate)
run: make test-cover
Expand Down Expand Up @@ -226,7 +226,7 @@ jobs:
run: npm run typecheck

# Vitest run with V8 coverage; exits non-zero if thresholds are not met.
# The threshold is defined in vite.config.ts (60 % — matches Go's minimum).
# The thresholds are defined in vite.config.ts (85 % lines / 82 % functions).
- name: Test with coverage gate (Vitest)
working-directory: web
run: npm run test:coverage
Expand Down Expand Up @@ -1009,9 +1009,12 @@ jobs:
# the same guard conformance, expr-oracle, ldap-integration and
# isolation-integration use.
#
# Linux only. The bash-joined variant gates on a linux worker and can run
# NOWHERE else; the Windows-gated variant has its own job below
# (ffmpeg-presets-windows), because it can run nowhere but a Windows runner.
# Linux is where the bash-joined variant's coverage is REQUIRED by name below.
# That variant no longer gates on an OS -- bash is not POSIX-only, git-bash
# puts it on Windows too -- so it also runs on the Windows job, but only
# opportunistically there (see that job's own note). The PowerShell-gated
# variant has its own job below (ffmpeg-presets-windows), because it can run
# nowhere but a Windows runner.
ffmpeg-presets:
name: ffmpeg reference presets (execution)
runs-on: ubuntu-latest
Expand Down Expand Up @@ -1104,10 +1107,20 @@ jobs:
go test -run TestFFmpegPreset -v -timeout 900s ./test/integration/ 2>&1 \
| tee /tmp/ffmpeg-presets-windows.log

# Only the PowerShell case is required by name. The rest either skip here
# (the bash variant gates on linux) or are already required by name on the
# Linux job, and duplicating them would make this job fail for reasons
# that have nothing to do with Windows.
# Only the PowerShell case is required by name. The rest are already
# required by name on the Linux job, and duplicating them would make this
# job fail for reasons that have nothing to do with Windows.
#
# TestFFmpegPreset_BashSegmentTranscodeJoins is the interesting omission.
# Its preset dropped its linux/macos gate (git-bash means bash is not
# POSIX-only), so it no longer skips here by construction -- it runs, and
# if its join fails on Windows the `go test` step above goes red, which is
# exactly the coverage that change needs. It is NOT required by name yet
# because it skips when `bash.exe` is not resolvable by Windows PATH
# lookup from the worker process, and that has not been observed on this
# runner image: requiring it now would turn the job red for an
# environmental reason rather than a real one. Promote it to the list once
# a run here is seen to PASS rather than SKIP.
- name: Assert the PowerShell join actually ran
shell: bash
run: |
Expand Down
4 changes: 2 additions & 2 deletions .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ builds:

archives:
- id: sqi
builds:
ids:
- sqi-server
- sqi-worker
name_template: >-
Expand All @@ -118,7 +118,7 @@ archives:
{{- else }}{{ .Arch }}{{ end }}
format_overrides:
- goos: windows
format: zip
formats: [zip]
files:
- LICENSE
- README.md
Expand Down
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,47 @@ Format follows [Conventional Commits](https://www.conventionalcommits.org/) and
> `docs/development.md`). It is refreshed during release prep and again by the
> release workflow, which bundles it into the release archives.

## [0.3.0] — 2026-08-18


### Bug Fixes

- Duplicate websocket push on subscribe ([#87](https://github.com/uberware/sqi/issues/87)) ([19b0607](https://github.com/uberware/sqi/commit/19b060770ff25a67dc94b2172e4b570a3353e008))
- Enforce task state machine on task status writes ([#97](https://github.com/uberware/sqi/issues/97)) ([9c386d4](https://github.com/uberware/sqi/commit/9c386d42c6f28a6ab9250a53916063fc1242b4c9))
- **web:** Upgrade react-router to 8.3.0 for GHSA-qwww-vcr4-c8h2 ([#101](https://github.com/uberware/sqi/issues/101)) ([09b3ac6](https://github.com/uberware/sqi/commit/09b3ac636b22f0d15674ee557157a9c8bdfef0c6))
- Atomic job submission ([#113](https://github.com/uberware/sqi/issues/113)) ([b1aa609](https://github.com/uberware/sqi/commit/b1aa609273526d11c92cd516934fbb6b6cf7b3fc))
- **scheduler:** Match attr.worker.os.family macos against a darwin worker ([#116](https://github.com/uberware/sqi/issues/116)) ([cdf0815](https://github.com/uberware/sqi/commit/cdf0815ef964a250d426d170855e566ba64f4f0a))
- **scheduler:** Resolve attr.worker.cpu.arch against a worker-reporte… ([#117](https://github.com/uberware/sqi/issues/117)) ([874c85f](https://github.com/uberware/sqi/commit/874c85fa29e22afd89a3eb447e1c18f97bc5aea3))


### Build

- Replace deprecated goreleaser archives keys so goreleaser check passes ([c5d75de](https://github.com/uberware/sqi/commit/c5d75de68e8bc7df23c5fd0c11e8e8529731d6c4))


### Documentation

- Correct project status and documentation drift ([#98](https://github.com/uberware/sqi/issues/98)) ([9026e68](https://github.com/uberware/sqi/commit/9026e68cba99bbb3cad386440c642d3b22e463af))
- Correct inaccurate and stale documentation across the repo ([5ac8fc4](https://github.com/uberware/sqi/commit/5ac8fc4a7aefd236ce52c8e7c22cd2c14ed27a18))
- Mark phase 3 and the EXPR extension as released in v0.3.0 ([8d6c278](https://github.com/uberware/sqi/commit/8d6c2782cc6b35ad6e003db6a9e641fbf664d263))


### Features

- Authentication infrastructure ([#80](https://github.com/uberware/sqi/issues/80)) ([15e72b6](https://github.com/uberware/sqi/commit/15e72b66037dca9c38fe108eeb91fe499d1bf7ae))
- Local accounts, login sessions, auth shell ([#81](https://github.com/uberware/sqi/issues/81)) ([a55e203](https://github.com/uberware/sqi/commit/a55e203cb3852c4f849d918853cc2bfb45b258e1))
- Api keys ([#82](https://github.com/uberware/sqi/issues/82)) ([7dcb0e0](https://github.com/uberware/sqi/commit/7dcb0e0dc49c043c517e09c81b699084f529aa7e))
- Role-based access control ([#83](https://github.com/uberware/sqi/issues/83)) ([3190510](https://github.com/uberware/sqi/commit/3190510f14c6055d5759ed33dd2de7b5f5058918))
- Job owner binding ([#84](https://github.com/uberware/sqi/issues/84)) ([8d7cf58](https://github.com/uberware/sqi/commit/8d7cf58c5c2b603eb31e2d9b6b4afb1b7167b387))
- Auth admin and testing ([#85](https://github.com/uberware/sqi/issues/85)) ([e1003da](https://github.com/uberware/sqi/commit/e1003daa5cdaf98acd8c4db12b1400894e1b52f4))
- LDAP and AD integration ([#86](https://github.com/uberware/sqi/issues/86)) ([4339e31](https://github.com/uberware/sqi/commit/4339e31bd086d0fcd3451dc38917b00662b5dc19))
- OIDC compatible SSO and standardization with LDAP support ([#95](https://github.com/uberware/sqi/issues/95)) ([842e198](https://github.com/uberware/sqi/commit/842e198031515da35df286bb5c7e4f860394a977))
- Task isolation run as user ([#99](https://github.com/uberware/sqi/issues/99)) ([785034d](https://github.com/uberware/sqi/commit/785034d7ece38ef23398f88cc05796c91c1cc839))
- Improved openJD conformance ([#100](https://github.com/uberware/sqi/issues/100)) ([2cdef4f](https://github.com/uberware/sqi/commit/2cdef4f78618cfdabad999d7559180c4a31ba585))
- Support the OpenJD EXPR extension ([#114](https://github.com/uberware/sqi/issues/114)) ([0682598](https://github.com/uberware/sqi/commit/068259857ae4c3557f47ebade8b17da11b33f402))
- Ffmpeg presets ([#115](https://github.com/uberware/sqi/issues/115)) ([91bb943](https://github.com/uberware/sqi/commit/91bb943281a5cc04c78220fc91819330e2eec364))
- Mistika render presets ([#118](https://github.com/uberware/sqi/issues/118)) ([fe14c99](https://github.com/uberware/sqi/commit/fe14c99ead5acc63e93bb29b9e59f20b19ba7b99))

## [0.2.0] — 2026-07-13


Expand Down
35 changes: 21 additions & 14 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,21 @@ We maintain a [roadmap and architecture guide](ROADMAP.md) organized by developm
git clone https://github.com/Uberware/sqi.git
cd sqi

# Go development (server and workers)
cd cmd/sqi-server
go mod download
go build
# Install the git hooks (gofumpt, goimports, go vet, golangci-lint, and the
# Conventional Commits check). Requires gofumpt, goimports and golangci-lint
# on your PATH — see docs/development.md for install commands.
make hooks

# Python development (client SDK and DCC submitters)
cd clients/python
pip install -e .
# Go development (server and workers) — builds the web UI bundle first and
# writes both binaries into ./bin/
make build

# Python development (client SDK)
make py-install

# Web UI development (TypeScript + React)
cd web
npm install
npm ci
npm run dev
```

Expand All @@ -43,13 +46,17 @@ npm run dev
- **Go**: Follow [Effective Go](https://golang.org/doc/effective_go). Run `make fmt` (gofumpt + goimports) and `make lint` (golangci-lint) before submitting
- **Python**: Follow [PEP 8](https://www.python.org/dev/peps/pep-0008/). Use type hints for all function signatures. Format and lint with ruff: `ruff format && ruff check --fix` — this handles style consistency automatically
- **TypeScript/React**: Use ESLint and Prettier with the project configuration. Functional components and hooks preferred
- **Commit messages**: Be clear and specific. Reference issue numbers where relevant (e.g., "Fix scheduler race condition in Phase 1 (#42)")
- **SPDX header**: every source file (Go, TypeScript, YAML, Python, shell) must carry `SPDX-License-Identifier: AGPL-3.0-or-later` before any `package`/`import`/module declaration, in the file's own comment syntax. See [`docs/spdx-header.md`](docs/spdx-header.md) for the exact template.
- **Commit messages**: [Conventional Commits](https://www.conventionalcommits.org) format — `type(scope)?: description` — is **enforced** by the `commit-msg` git hook installed by `make hooks`. Valid types: `feat fix docs style refactor test chore build ci perf revert`. Reference issue numbers in the description where relevant (e.g. `fix(scheduler): correct heartbeat timeout calculation (#42)`). `CHANGELOG.md` is generated from these messages by git-cliff, so a non-conforming commit never reaches the changelog.

**Testing:**

- Unit tests are required for code changes. The enforced coverage floor is 70% (`COVERAGE_MIN` in the Makefile); aim higher on new code
- Integration tests are encouraged for complex features
- Integration tests are encouraged for complex features. They live in `test/integration/`; those behind the `integration` build tag need `make test-integration`, because they depend on something the default suite should not require (a built binary, or an external service)
- Run `make test` (or `make ci`) before submitting a PR — do not run bare `go test ./...` from the repo root, since `web/node_modules/` contains third-party Go files the Makefile filters out
- **Changing LDAP code?** Run `make test-ldap`. It drives the login path against a real OpenLDAP server in a throwaway container, which is the only thing that catches a mistake in how sqi talks to a directory *on the wire*
- **Changing OIDC/SSO code?** Run `make test-oidc`. It drives the whole browser flow against a real Keycloak in a throwaway container, which is the only thing that catches what a real provider *omits* — most importantly a missing group claim, which validates fine and silently drops every user to `default_role`
- Both need Docker and **skip** without it — and a skip verifies nothing, so confirm the tests actually ran rather than trusting the exit code. Details in [`docs/development.md`](docs/development.md#testing-against-a-real-directory-or-identity-provider)

**Submitting a PR:**

Expand Down Expand Up @@ -92,8 +99,8 @@ Conventions:
- Mock at the network boundary (the `apiFetch`/query layer), not at the component
internals, so tests exercise real component wiring.
- New components and hooks ship with tests; coverage is enforced against the
threshold in `web/vite.config.ts`. See the existing `DataTable.test.tsx`,
`StatusBadge.test.tsx`, and the `src/api` / `src/hooks` tests for the patterns.
threshold in `web/vite.config.ts`. See the existing `StatusBadge.test.tsx`,
`Pagination.test.tsx`, and the `src/api` / `src/hooks` tests for the patterns.

**API client pattern.** All server access goes through `src/api/` — never call
`fetch` directly from a component.
Expand Down Expand Up @@ -207,8 +214,8 @@ The [ROADMAP.md](ROADMAP.md) document outlines development phases. Current prior

- **Phase 1** (v0.1 — released): Core scheduler, pull-based workers, basic web UI, OpenJD execution
- **Phase 2** (v0.2 — released): Product system, preset library, DCC submitters
- **Phase 3** (next): Auth (LDAP, OAuth2), multi-user role model
- **Phase 4** (planned): Production hardening, PostgreSQL, HA, auto-scaling
- **Phase 3** (v0.3 — released): Auth (local accounts, API keys, RBAC, LDAP/AD, OAuth2/OIDC SSO), multi-user role model, and run-as-user task isolation; v0.3 also ships the OpenJD `EXPR` extension and expanded ffmpeg and Mistika reference presets
- **Phase 4** (next): Production hardening, PostgreSQL, HA, auto-scaling

Code contributions aligned with the current phase are most likely to be accepted quickly. Contributions targeting later phases are welcome but may take longer to review if they require design discussion.

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

📖 **[Documentation](https://uberware.github.io/sqi/)** — quickstart, architecture, configuration, and full reference.

> **Status:** v0.2.0 (Phase 2) released. Builds on the Phase 1 core with products and presets as an authoring layer over OpenJD, a community preset-library integration, product-driven submission, additional path-translation modes, S3-compatible storage, compute locations, and in-application DCC submitters for Maya, Houdini, Nuke, and Blender. **Phase 3** (opt-in auth & multi-user) — identity complete, task isolation in progress. Contributions, feedback, and discussion welcome.
> **Status:** v0.3.0 (Phase 3) released — opt-in authentication and multi-user support: local accounts, API keys, RBAC, job-owner identity, LDAP/AD, OAuth2/OIDC SSO, and queue-scoped run-as-user task isolation on POSIX and Windows. Also in this release: the official OpenJD `EXPR` expression-language extension, and expanded reference presets (ffmpeg transcode, sequence-encode and segment-transcode; Mistika Boutique, VR and Workflows). Auth is **off by default**, so an existing deployment is unaffected until enabled. **Phase 4** (production hardening) is next. Contributions, feedback, and discussion welcome.

---

Expand All @@ -24,7 +24,7 @@ The render farm management space is in an awkward moment. Legacy on-premises sys

**Not tied to any cloud provider.** Workers run on Linux, macOS, and Windows — bare metal, VMs, or containers. Cloud compute locations are supported across AWS, GCP, Azure, and any provider that can run a container or a binary. Your control plane runs where you want it.

**OpenJD compatible.** `sqi` adopts the [Open Job Description](https://github.com/OpenJobDescription/openjd-specifications) format as its native job execution layer — not an import step or a partial translation, but the format `sqi` jobs are authored and validated in directly. This is a real standard designed for portability, not a proprietary format. Conformance is measured, not asserted: `sqi` runs the official OpenJD conformance test suite on every CI build and publishes the current pass rate and known gaps at [`docs/openjd-conformance.md`](docs/openjd-conformance.md). One caveat is by design rather than a gap: a template that opts into an OpenJD extension `sqi` does not implement — such as `EXPR` — is rejected outright, rather than being accepted and misinterpreted.
**OpenJD compatible.** `sqi` adopts the [Open Job Description](https://github.com/OpenJobDescription/openjd-specifications) format as its native job execution layer — not an import step or a partial translation, but the format `sqi` jobs are authored and validated in directly. This is a real standard designed for portability, not a proprietary format. Conformance is measured, not asserted: `sqi` runs the official OpenJD conformance test suite on every CI build and publishes the current pass rate and known gaps at [`docs/openjd-conformance.md`](docs/openjd-conformance.md). One caveat is by design rather than a gap: a template that opts into an OpenJD extension `sqi` does not implement is rejected outright, rather than being accepted and misinterpreted. The official `EXPR` (expression language) extension **is** implemented and supported — see [`docs/openjd-extensions/expr.md`](docs/openjd-extensions/expr.md).

**General purpose.** Rendering is the primary use case and the domain `sqi` is designed around, but the job model is general. Any workload expressible as a command with defined inputs, outputs, and environment is a valid `sqi` job — simulation, transcoding, machine learning pipelines, data processing, software development, or anything else a studio runs at scale.

Expand Down Expand Up @@ -81,7 +81,7 @@ Both modes run the same software. The difference is configuration.

## Status and roadmap

`sqi` v0.1.0 delivered the Phase 1 core: scheduler, pull-based workers, OpenJD job execution, and a basic web UI, with a Python client and single-binary or Docker Compose deployment. v0.2.0 completes Phase 2 — products and presets as an authoring layer over OpenJD, the community preset-library integration, a product-driven submission form, additional path-translation modes, S3-compatible storage, compute locations, and in-application DCC submitters for Maya, Houdini, Nuke, and Blender. **Phase 3** — opt-in authentication and multi-user support: local accounts and API keys, role-based access control, an authenticated owner/submitter identity on jobs, LDAP/Active Directory, OAuth2/OIDC SSO, and queue-scoped run-as-user task isolation (tasks execute as a distinct, unprivileged OS account instead of the worker's own, on POSIX and Windows) — is complete and merged on `main`, unreleased. Auth is off by default, so an existing deployment is unaffected until enabled. Production-hardening features (Phase 4) follow.
`sqi` v0.1.0 delivered the Phase 1 core: scheduler, pull-based workers, OpenJD job execution, and a basic web UI, with a Python client and single-binary or Docker Compose deployment. v0.2.0 completes Phase 2 — products and presets as an authoring layer over OpenJD, the community preset-library integration, a product-driven submission form, additional path-translation modes, S3-compatible storage, compute locations, and in-application DCC submitters for Maya, Houdini, Nuke, and Blender. **Phase 3** — opt-in authentication and multi-user support: local accounts and API keys, role-based access control, an authenticated owner/submitter identity on jobs, LDAP/Active Directory, OAuth2/OIDC SSO, and queue-scoped run-as-user task isolation (tasks execute as a distinct, unprivileged OS account instead of the worker's own, on POSIX and Windows) — ships in v0.3.0, alongside the official OpenJD `EXPR` expression-language extension and expanded ffmpeg and Mistika reference presets. Auth is off by default, so an existing deployment is unaffected until enabled. Production-hardening features (Phase 4) follow.

This is a real project with a concrete development commitment, not a design document waiting for funding. Feedback on priorities is welcome — [open an issue](https://github.com/uberware/sqi/issues/new) or [start a discussion](https://github.com/uberware/sqi/discussions/new/choose).

Expand Down
Loading