Skip to content
cerulean-worksPublic

About

Cerulean's monorepo for model training, evaluation, and development tooling.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

Cerulean Labs Flock banner


Flock is Cerulean's workspace for model evaluation, experiment tracking and shared dataset and checkpoint storage.

Documentation

Read the Flock docs for task guides and references grouped by CLI, Evals and App. To build or preview the site locally, see the documentation contributor guide.

Getting Started

Installing dependencies

Install Python 3.14 and uv, then run the commands below from the repository root. This installs all module dependencies and development tools, with Flock in editable mode so source changes take effect without reinstalling.

uv sync --all-packages --locked
uv run --all-packages python -c "import flock.training; import flock.eval; import flock.cli; import app.api.app"

Adding new dependencies

Add each dependency to the module that uses it with uv add --package, choosing the package name from the commands below. Each module keeps its dependencies in its own pyproject.toml, while the workspace shares one lockfile and one .venv. For shared development tools, use uv add --group dev <dependency>.

uv add --package flock-training <dependency>
uv add --package flock-eval <dependency>
uv add --package flock-cli <dependency>
uv add --package flock-app-api <dependency>
uv add --package flock-tests <dependency>

Running tests

For the full test suite, worker limits, CPU-only CI dependencies, and timing reports, see Running Flock's tests.

Modules

CLI

flock.cli provides account and storage access. After installing the workspace, run uv run flock login to authorize this machine with Cerulean's service and configure the wandb SDK. Sign-in and access explains the credentials; S3 storage explains key types and renewal before connecting your tools.

Evaluation

flock.eval scores pinned language models on tasks and suites, writes metrics and per-question predictions, and can score a model held in a training process. Evaluate a model introduces tasks, metrics and comparable results before walking through flock eval.

App

app/api serves the wandb protocol, the read API and organization storage; app/web is the dashboard. Start with Track an experiment or Explore experiments. For local development and separate installations, see Run locally and Deploy Flock.

Training

flock.training runs single-device, next-token training on Lance text datasets. Train with Flock explains the example experiment and configuration. To evaluate a model during training, use the in-memory evaluation API.

About

Cerulean's monorepo for model training, evaluation, and development tooling.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages