Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
uses: erlef/setup-beam@v1
with:
otp-version: "28"
gleam-version: "1.13.0"
gleam-version: "1.14.0"
rebar3-version: "3"

- name: Download dependencies
Expand Down
31 changes: 30 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [2.1.0] - 2026-01-31

### Added

- **Runner-level hooks** (`dream_test/runner`, `dream_test/parallel`)

- New `before_each_test`, `after_each_test`, `before_each_suite`, `after_each_suite`,
`before_all_suites`, and `after_all_suites` hooks
- Hooks receive structured metadata via `types.TestInfo` and `types.SuiteInfo`
- Per-test hooks run in the executor with sandboxing and correct ordering around suite hooks

- **Test/suite metadata** (`dream_test/types`)

- New `TestInfo` and `SuiteInfo` types, including best-effort `source`
- Source tracking for unit discovery (module name) and Gherkin discovery (feature path)

### Changed

- **Setup failure status** (`dream_test/parallel`)

- Runner `before_each_test` failures now yield `SetupFailed` and skip test bodies

### Documentation

- Added runner hook section to the runner guide with a tested snippet
- Updated README and documentation index to highlight runner hooks
- Refreshed compatibility reports (`COMPATIBILITY.md`, `COMPATIBILITY_REPORT.md`, `CONSTRAINT_ANALYSIS.md`)

## [2.0.0] - 2025-12-27

### Added
Expand Down Expand Up @@ -283,7 +311,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- STANDARDS document for code conventions
- API documentation for all public modules

[Unreleased]: https://github.com/TrustBound/dream_test/compare/2.0.0...HEAD
[Unreleased]: https://github.com/TrustBound/dream_test/compare/2.1.0...HEAD
[2.1.0]: https://github.com/TrustBound/dream_test/compare/2.0.0...2.1.0
[2.0.0]: https://github.com/TrustBound/dream_test/compare/1.2.0...2.0.0
[1.2.0]: https://github.com/TrustBound/dream_test/compare/1.1.0...1.2.0
[1.1.0]: https://github.com/TrustBound/dream_test/compare/1.0.3...1.1.0
Expand Down
6 changes: 3 additions & 3 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
| Package | Minimum | Maximum | Status |
|--------------|---------|---------|--------|
| gleam_erlang | 1.0.0 | 1.3.0 | Tested |
| gleam_json | 3.0.1 | 3.1.0 | Tested |
| gleam_json | 3.0.2 | 3.1.0 | Tested |
| gleam_otp | 1.1.0 | 1.2.0 | Tested |
| gleam_regexp | 1.0.0 | 1.1.1 | Tested |
| gleam_stdlib | 0.60.0 | 0.67.1 | Tested |
| gleam_stdlib | 0.60.0 | 0.68.1 | Tested |

All versions tested with: `gleam test`

Last tested: 2025-12-27
Last tested: 2026-02-01
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
gleam add --dev dream_test
```

Current version: `2.1.0`

## Why Dream Test?

Rapid application development needs testing tools that scale and support the growing needs of the application without slowing down progress. Dream test was designed to help engineers write expressive unit and integration tests for their applications using the tools and techniques they know from other ecosystems; adapted properly to gleam and the beam.
Expand Down Expand Up @@ -60,6 +62,7 @@ Rapid application development needs testing tools that scale and support the gro
| ⏱️ **Timeouts** | Per-test timeout control |
| 🔍 **Test discovery** | Find tests from file paths |
| 🚨 **Exit-on-failure** | Fail fast for CI |
| 🪝 **Runner hooks** | Runner-level hooks for per-test/per-suite/per-run setup and teardown |
| 🧩 **Suite-specific execution config** | Run some suites sequential/with custom timeouts in the same runner (`runner.add_suites_with_config(...)`) |

### Reporting
Expand All @@ -84,7 +87,7 @@ Dream Test splits reporting into:
4. [Context-Aware Tests](documentation/04-context-aware-tests.md)
5. [Assertions & Matchers](documentation/05-assertions-and-matchers.md)
6. [Lifecycle Hooks](documentation/06-lifecycle-hooks.md)
7. [Runner & Execution](documentation/07-runner-and-execution.md)
7. [Runner & Execution](documentation/07-runner-and-execution.md) — includes runner hooks
8. [Reporters](documentation/08-reporters.md)
9. [Snapshot Testing](documentation/09-snapshot-testing.md)
10. [Gherkin BDD](documentation/10-gherkin-bdd.md)
Expand Down
2 changes: 2 additions & 0 deletions documentation/02-quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ That explicitness is the source of most of Dream Test’s reliability: when a te

If you’re looking for a mental model of “how does the runner find my tests?”: in Dream Test, **`main()` chooses what runs** by passing suites to the runner. You can list suites explicitly, or generate the list via discovery.

If you need setup/teardown that spans multiple suites, see the runner-level hooks section in [Runner & execution model](07-runner-and-execution.md).

### Choose your first runner style

There are two good starting points:
Expand Down
2 changes: 2 additions & 0 deletions documentation/03-writing-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ Dream Test’s unit DSL is built for that style, but there’s a deeper design g
- **Your test module stays ordinary Gleam** (no hidden discovery side-effects).
- **Failures should read well** (because tests are communication, not just verification).

If you need setup/teardown that spans multiple suites, use runner-level hooks in [Runner & execution model](07-runner-and-execution.md).

### `describe` + `it` (the core loop)

```gleam
Expand Down
2 changes: 2 additions & 0 deletions documentation/04-context-aware-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ This is the right tool when:

If you don’t need an explicit context, prefer `dream_test/unit` — it’s simpler.

For setup/teardown that spans multiple suites, use runner-level hooks in [Runner & execution model](07-runner-and-execution.md).

### The idea: context flows through the suite

- You give `describe` an initial `seed` value.
Expand Down
2 changes: 2 additions & 0 deletions documentation/06-lifecycle-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ Hooks are a power tool: they remove repetition, but they can also hide the story

The goal of Dream Test’s hook design is to keep hooks **predictable and debuggable**, especially under parallel execution.

For runner-level hooks (per-test/per-suite/per-run), see [Runner & execution](07-runner-and-execution.md).

### Mental model

Hooks are part of the nested test structure that the runner executes around tests:
Expand Down
80 changes: 80 additions & 0 deletions documentation/07-runner-and-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,86 @@ pub fn main() {

<sub>🧪 [Tested source](../examples/snippets/test/snippets/runner/suite_specific_config.gleam)</sub>

### Runner hooks (test/suite/run)

Runner hooks let you apply setup/teardown **across suites** without modifying
each suite’s contents. Hooks are metadata-aware and receive structured info
about what is running.

**`types.TestInfo` fields**:

- `name`: leaf test name (`it("...")`)
- `full_name`: group path + test name (e.g. `["outer", "inner", "leaf"]`)
- `tags`: effective tags (includes inherited group tags)
- `kind`: `types.TestKind` (Unit, GherkinScenario(id), etc.)
- `source`: best-effort origin string (see below)

**`types.SuiteInfo` fields**:

- `name`: top-level suite/group name
- `tests`: deterministic list of `TestInfo` values that will execute
- `source`: best-effort origin string (see below)

**Hook signatures**:

- `before_each_test` / `after_each_test` receive `types.TestInfo`.
- `before_each_suite` / `after_each_suite` receive `types.SuiteInfo`.
- `before_all_suites` / `after_all_suites` receive the full list of `SuiteInfo` after filtering.

`source` is best-effort:

- Unit discovery suites use the **module name**.
- Gherkin discovery suites use the **.feature path**.
- Manually constructed suites use `None`.

```gleam
import dream_test/matchers.{succeed}
import dream_test/runner
import dream_test/types.{type SuiteInfo, type TestInfo}
import dream_test/unit.{describe, it}
import gleam/int
import gleam/io
import gleam/list
import gleam/string

pub fn tests() {
describe("Example", [
it("a", fn() { Ok(succeed()) }),
it("b", fn() { Ok(succeed()) }),
])
}

pub fn main() {
runner.new([tests()])
|> runner.before_all_suites(before_all_suites_hook)
|> runner.before_each_suite(before_each_suite_hook)
|> runner.before_each_test(before_each_test_hook)
|> runner.after_each_test(after_each_test_hook)
|> runner.run()
}

fn before_all_suites_hook(suites: List(SuiteInfo)) -> Result(Nil, String) {
io.println("suites: " <> int.to_string(list.length(suites)))
Ok(Nil)
}

fn before_each_suite_hook(suite: SuiteInfo) -> Result(Nil, String) {
io.println("suite: " <> suite.name)
Ok(Nil)
}

fn before_each_test_hook(info: TestInfo, ctx: Nil) -> Result(Nil, String) {
io.println("test: " <> string.join(info.full_name, " > "))
Ok(ctx)
}

fn after_each_test_hook(_info: TestInfo, _ctx: Nil) -> Result(Nil, String) {
Ok(Nil)
}
```

<sub>🧪 [Tested source](../examples/snippets/test/snippets/runner/runner_hooks.gleam)</sub>

### Choosing a concurrency number (practical guidance)

- Start with the default.
Expand Down
2 changes: 1 addition & 1 deletion documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ That’s why you’ll see the same patterns repeated:
4. [Context-aware unit tests](04-context-aware-tests.md) — when “setup returns a value” and you need to pass it into tests.
5. [Assertions & matchers](05-assertions-and-matchers.md) — how and why the `should` pipeline works.
6. [Lifecycle hooks](06-lifecycle-hooks.md) — power tools, and how to keep them from hiding meaning.
7. [Runner & execution model](07-runner-and-execution.md) — concurrency, timeouts, CI, and reliability.
7. [Runner & execution model](07-runner-and-execution.md) — concurrency, timeouts, CI, reliability, and runner hooks.
8. [Reporters](08-reporters.md) — humans vs machines, streaming vs post-run.
9. [Snapshot testing](09-snapshot-testing.md) — when snapshots make tests clearer (and when they make them worse).
10. [Gherkin / BDD](10-gherkin-bdd.md) — scenario testing with a world state and placeholder captures.
Expand Down
5 changes: 3 additions & 2 deletions examples/failure_showcase/manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@
# You typically do not need to edit this file

packages = [
{ name = "dream_test", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_json", "gleam_otp", "gleam_regexp", "gleam_stdlib"], source = "local", path = "../.." },
{ name = "dream_test", version = "2.1.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_json", "gleam_otp", "gleam_regexp", "gleam_stdlib"], source = "local", path = "../.." },
{ name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" },
{ name = "gleam_json", version = "3.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_json", source = "hex", outer_checksum = "44FDAA8847BE8FC48CA7A1C089706BD54BADCC4C45B237A992EDDF9F2CDB2836" },
{ name = "gleam_otp", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_stdlib"], otp_app = "gleam_otp", source = "hex", outer_checksum = "BA6A294E295E428EC1562DC1C11EA7530DCB981E8359134BEABC8493B7B2258E" },
{ name = "gleam_regexp", version = "1.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_regexp", source = "hex", outer_checksum = "9C215C6CA84A5B35BB934A9B61A9A306EC743153BE2B0425A0D032E477B062A9" },
{ name = "gleam_stdlib", version = "0.67.1", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "6CE3E4189A8B8EC2F73AB61A2FBDE49F159D6C9C61C49E3B3082E439F260D3D0" },
{ name = "gleam_stdlib", version = "0.68.1", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "F7FAEBD8EF260664E86A46C8DBA23508D1D11BB3BCC6EE1B89B3BC3E5C83FF1E" },
]

[requirements]
dream_test = { path = "../.." }
gleam_erlang = { version = ">= 1.3.0 and < 2.0.0" }
gleam_stdlib = { version = ">= 0.44.0 and < 2.0.0" }
6 changes: 3 additions & 3 deletions examples/snippets/manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
# You typically do not need to edit this file

packages = [
{ name = "dream_test", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_json", "gleam_otp", "gleam_regexp", "gleam_stdlib"], source = "local", path = "../.." },
{ name = "dream_test", version = "2.1.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_json", "gleam_otp", "gleam_regexp", "gleam_stdlib"], source = "local", path = "../.." },
{ name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" },
{ name = "gleam_json", version = "3.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_json", source = "hex", outer_checksum = "44FDAA8847BE8FC48CA7A1C089706BD54BADCC4C45B237A992EDDF9F2CDB2836" },
{ name = "gleam_otp", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_stdlib"], otp_app = "gleam_otp", source = "hex", outer_checksum = "BA6A294E295E428EC1562DC1C11EA7530DCB981E8359134BEABC8493B7B2258E" },
{ name = "gleam_regexp", version = "1.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_regexp", source = "hex", outer_checksum = "9C215C6CA84A5B35BB934A9B61A9A306EC743153BE2B0425A0D032E477B062A9" },
{ name = "gleam_stdlib", version = "0.67.1", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "6CE3E4189A8B8EC2F73AB61A2FBDE49F159D6C9C61C49E3B3082E439F260D3D0" },
{ name = "gleam_stdlib", version = "0.68.1", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "F7FAEBD8EF260664E86A46C8DBA23508D1D11BB3BCC6EE1B89B3BC3E5C83FF1E" },
]

[requirements]
dream_test = { path = "../.." }
gleam_erlang = { version = ">= 1.3.0 and < 2.0.0" }
gleam_otp = { version = ">= 1.2.0 and < 2.0.0" }
gleam_regexp = { version = ">= 1.0.0 and < 2.0.0" }
gleam_stdlib = { version = ">= 0.44.0 and < 2.0.0" }
gleam_otp = { version = ">= 1.2.0 and < 2.0.0" }
2 changes: 2 additions & 0 deletions examples/snippets/test/snippets/gherkin/gherkin_types.gleam
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ pub fn tests() {

Feature(
name: "Example feature",
source: None,
description: None,
tags: [],
background: None,
Expand All @@ -91,6 +92,7 @@ pub fn tests() {
|> be_equal(
Feature(
name: "Example feature",
source: None,
description: None,
tags: [],
background: None,
Expand Down
43 changes: 43 additions & 0 deletions examples/snippets/test/snippets/runner/runner_hooks.gleam
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import dream_test/matchers.{succeed}
import dream_test/runner
import dream_test/types.{type SuiteInfo, type TestInfo}
import dream_test/unit.{describe, it}
import gleam/int
import gleam/io
import gleam/list
import gleam/string

pub fn tests() {
describe("Example", [
it("a", fn() { Ok(succeed()) }),
it("b", fn() { Ok(succeed()) }),
])
}

pub fn main() {
runner.new([tests()])
|> runner.before_all_suites(before_all_suites_hook)
|> runner.before_each_suite(before_each_suite_hook)
|> runner.before_each_test(before_each_test_hook)
|> runner.after_each_test(after_each_test_hook)
|> runner.run()
}

fn before_all_suites_hook(suites: List(SuiteInfo)) -> Result(Nil, String) {
io.println("suites: " <> int.to_string(list.length(suites)))
Ok(Nil)
}

fn before_each_suite_hook(suite: SuiteInfo) -> Result(Nil, String) {
io.println("suite: " <> suite.name)
Ok(Nil)
}

fn before_each_test_hook(info: TestInfo, ctx: Nil) -> Result(Nil, String) {
io.println("test: " <> string.join(info.full_name, " > "))
Ok(ctx)
}

fn after_each_test_hook(_info: TestInfo, _ctx: Nil) -> Result(Nil, String) {
Ok(Nil)
}
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ pub fn main() {
write: write,
total: total,
completed: completed,
runner_before_each_test: [],
runner_after_each_test: [],
),
)
let parallel.RunRootParallelWithReporterResult(
Expand Down
8 changes: 4 additions & 4 deletions gleam.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
name = "dream_test"
version = "2.0.0"
version = "2.1.0"
description = "A testing framework for Gleam that gets out of your way"
licences = ["MIT"]
repository = { type = "github", user = "TrustBound", repo = "dream_test" }
links = [{ title = "Documentation", href = "https://hexdocs.pm/dream_test" }]

[dependencies]
gleam_stdlib = ">= 0.60.0 and < 1.0.0"
gleam_otp = ">= 1.1.0 and < 2.0.0"
gleam_erlang = ">= 1.0.0 and < 2.0.0"
gleam_json = ">= 3.0.2 and < 4.0.0"
gleam_otp = ">= 1.1.0 and < 2.0.0"
gleam_regexp = ">= 1.0.0 and < 2.0.0"
gleam_json = ">= 3.0.1 and < 4.0.0"
gleam_stdlib = ">= 0.60.0 and < 1.0.0"

[dev-dependencies]
6 changes: 3 additions & 3 deletions manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ packages = [
{ name = "gleam_json", version = "3.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_json", source = "hex", outer_checksum = "44FDAA8847BE8FC48CA7A1C089706BD54BADCC4C45B237A992EDDF9F2CDB2836" },
{ name = "gleam_otp", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_erlang", "gleam_stdlib"], otp_app = "gleam_otp", source = "hex", outer_checksum = "BA6A294E295E428EC1562DC1C11EA7530DCB981E8359134BEABC8493B7B2258E" },
{ name = "gleam_regexp", version = "1.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_regexp", source = "hex", outer_checksum = "9C215C6CA84A5B35BB934A9B61A9A306EC743153BE2B0425A0D032E477B062A9" },
{ name = "gleam_stdlib", version = "0.67.0", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "6368313DB35963DC02F677A513BB0D95D58A34ED0A9436C8116820BF94BE3511" },
{ name = "gleam_stdlib", version = "0.68.1", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "F7FAEBD8EF260664E86A46C8DBA23508D1D11BB3BCC6EE1B89B3BC3E5C83FF1E" },
]

[requirements]
gleam_erlang = { version = ">= 1.0.0 and < 2.0.0" }
gleam_json = { version = ">= 3.0.1 and < 4.0.0" }
gleam_json = { version = ">= 3.0.2 and < 4.0.0" }
gleam_otp = { version = ">= 1.1.0 and < 2.0.0" }
gleam_regexp = { version = ">= 1.0.0 and < 2.0.0" }
gleam_stdlib = { version = ">= 0.60.0 and < 2.0.0" }
gleam_stdlib = { version = ">= 0.60.0 and < 1.0.0" }
Loading