Skip to content

Commit 8dd1663

Browse files
h4x0rclaude
h4x0r
and
claude
committed
feat: add CI, e2e tests with DFTT fixtures, 98.8% unit test coverage
- Add GitHub Actions CI (fmt, clippy, test, cargo-deny) with pinned action SHAs - Add pre-commit hooks (fmt, clippy, test, cargo-deny) - Add 3 small E01 test fixtures from Digital Corpora (~1.2 MB total) - Add 10 e2e tests verifying full-media MD5 against libewf and Sleuth Kit - Add 12 unit tests covering error paths and edge cases (98.8% coverage) - Fix clippy warnings (uninlined_format_args, redundant_slicing, io_other_error) - Remove "How it works" section from README (internal implementation detail) - Update VALIDATION.md with all 6 validated images Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 6cda6ad commit 8dd1663

14 files changed

Lines changed: 979 additions & 98 deletions

‎.github/workflows/ci.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
env:
10+
CARGO_TERM_COLOR: always
11+
RUSTFLAGS: -Dwarnings
12+
13+
jobs:
14+
fmt:
15+
name: Format
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
19+
- uses: dtolnay/rust-toolchain@stable
20+
with:
21+
components: rustfmt
22+
- run: cargo fmt --check
23+
24+
clippy:
25+
name: Clippy
26+
runs-on: ubuntu-latest
27+
steps:
28+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
29+
- uses: dtolnay/rust-toolchain@stable
30+
with:
31+
components: clippy
32+
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
33+
- run: cargo clippy --all-targets -- -D warnings
34+
35+
test:
36+
name: Test
37+
runs-on: ubuntu-latest
38+
steps:
39+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
40+
- uses: dtolnay/rust-toolchain@stable
41+
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
42+
- run: cargo test --lib --test e2e_dftt
43+
44+
deny:
45+
name: Cargo Deny
46+
runs-on: ubuntu-latest
47+
steps:
48+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
49+
- uses: EmbarkStudios/cargo-deny-action@34899fc7ba81ca6268d5947a7a16b4649013fea1 # v2.0.11

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
11
/target
2+
.DS_Store
3+
tarpaulin-report.json

‎.pre-commit-config.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
repos:
2+
- repo: local
3+
hooks:
4+
- id: cargo-fmt
5+
name: cargo fmt
6+
entry: cargo fmt --check
7+
language: system
8+
types: [rust]
9+
pass_filenames: false
10+
11+
- id: cargo-clippy
12+
name: cargo clippy
13+
entry: cargo clippy --all-targets -- -D warnings
14+
language: system
15+
types: [rust]
16+
pass_filenames: false
17+
18+
- id: cargo-test
19+
name: cargo test (unit)
20+
entry: cargo test --lib --test e2e_dftt
21+
language: system
22+
types: [rust]
23+
pass_filenames: false
24+
25+
- id: cargo-deny
26+
name: cargo deny
27+
entry: cargo deny check
28+
language: system
29+
files: (Cargo\.toml|Cargo\.lock|deny\.toml)
30+
pass_filenames: false

‎README.md‎

Lines changed: 1 addition & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -90,33 +90,9 @@ let ntfs = Ntfs::new(&mut reader)?;
9090
| L01/Lx01 (logical) | Not yet |
9191
| S01 (SMART) | Not yet |
9292

93-
## How it works
94-
95-
EWF stores disk data as zlib-compressed 32 KB chunks across one or more segment files. Each segment contains a linked list of section descriptors pointing to volume geometry, chunk offset tables, and compressed data.
96-
97-
```
98-
.E01 file layout:
99-
┌─────────────┐
100-
│ File Header │ 13 bytes: EVF signature + segment number
101-
├─────────────┤
102-
│ Section │ 76 bytes each, linked list:
103-
│ Descriptors │ volume → table → sectors → done
104-
├─────────────┤
105-
│ Volume │ Chunk geometry (sectors/chunk, bytes/sector)
106-
├─────────────┤
107-
│ Table │ Chunk offset array (4 bytes per entry)
108-
├─────────────┤
109-
│ Sectors │ Compressed chunk data
110-
├─────────────┤
111-
│ done │ End of chain
112-
└─────────────┘
113-
```
114-
115-
`EwfReader::open()` walks each segment's section chain, builds a flat `Vec<Chunk>` index, then serves `Read + Seek` by mapping any byte offset to its chunk in O(1).
116-
11793
## Validation
11894

119-
Full-media MD5 comparison against libewf and The Sleuth Kit confirms bit-identical output across 3 public forensic images totaling 303 GiB. See [docs/VALIDATION.md](docs/VALIDATION.md) for results, image sources, and reproduction steps.
95+
Full-media MD5 comparison against libewf and The Sleuth Kit confirms bit-identical output across 6 public forensic images (303+ GiB of media). Three small images from [Digital Corpora](https://digitalcorpora.org/) are committed as test fixtures and run in CI. See [docs/VALIDATION.md](docs/VALIDATION.md) for results, image sources, and reproduction steps.
12096

12197
## Acknowledgments
12298

‎deny.toml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
[advisories]
2+
version = 2
3+
ignore = []
4+
5+
[licenses]
6+
version = 2
7+
allow = [
8+
"MIT",
9+
"Apache-2.0",
10+
"BSD-2-Clause",
11+
"BSD-3-Clause",
12+
"ISC",
13+
"Unicode-3.0",
14+
"Zlib",
15+
]
16+
17+
[bans]
18+
multiple-versions = "warn"
19+
wildcards = "deny"
20+
21+
[sources]
22+
unknown-registry = "deny"
23+
unknown-git = "deny"
24+
allow-registry = ["https://github.com/rust-lang/crates.io-index"]
25+
allow-git = []

‎docs/VALIDATION.md‎

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,58 @@ Every byte of decompressed media is hashed and compared — not sampled.
6969

7070
**Full-media MD5:** `522df9db8289f4f8132cf47b14d20fb8` — identical across libewf, Sleuth Kit, and ewf crate.
7171

72+
### 4. exfat1 (DFTT)
73+
74+
| Property | Value |
75+
|----------|-------|
76+
| Project | [Digital Forensics Tool Testing (DFTT)](http://dftt.sourceforge.net/) (Brian Carrier) |
77+
| Source | [Digital Corpora](https://digitalcorpora.org/) — AWS Open Data |
78+
| URL | `https://digitalcorpora.s3.amazonaws.com/corpora/drives/dftt-2004/exfat1.E01` |
79+
| Filename | `exfat1.E01` |
80+
| E01 file size | 274,722 bytes (268 KB) |
81+
| E01 MD5 | `74aca823a3959867a9de72a6b4c79b50` |
82+
| Format | EnCase 6, deflate best-compression |
83+
| Media size | 100,020,736 bytes (95 MiB) |
84+
| Sectors/chunk | 64 |
85+
| Filesystem | exFAT |
86+
87+
**Full-media MD5:** `0777ee90c27ed5ff5868af2015bed635` — identical across libewf, Sleuth Kit, and ewf crate.
88+
89+
### 5. imageformat_mmls_1 (DFTT)
90+
91+
| Property | Value |
92+
|----------|-------|
93+
| Project | [Digital Forensics Tool Testing (DFTT)](http://dftt.sourceforge.net/) (Brian Carrier) |
94+
| Source | [Digital Corpora](https://digitalcorpora.org/) — AWS Open Data |
95+
| URL | `https://digitalcorpora.s3.amazonaws.com/corpora/drives/dftt-2004/imageformat_mmls_1.E01` |
96+
| Filename | `imageformat_mmls_1.E01` |
97+
| E01 file size | 414,941 bytes (405 KB) |
98+
| E01 MD5 | `bb6c6bec25d589e87a11af9129275cc9` |
99+
| Format | FTK Imager, no compression |
100+
| Media size | 62,915,072 bytes (60 MiB) |
101+
| Sectors/chunk | 64 |
102+
| Filesystem | NTFS (partition at offset 65536) |
103+
| Description | Created to test Sleuth Kit libraries |
104+
105+
**Full-media MD5:** `8ec671e301095c258224aad701740503` — identical across libewf, Sleuth Kit, and ewf crate.
106+
107+
### 6. nps-2010-emails (NPS)
108+
109+
| Property | Value |
110+
|----------|-------|
111+
| Project | Naval Postgraduate School (NPS) forensic test corpora |
112+
| Source | [Digital Corpora](https://digitalcorpora.org/) — AWS Open Data |
113+
| URL | `https://digitalcorpora.s3.amazonaws.com/corpora/drives/nps-2010-emails/nps-2010-emails.E01` |
114+
| Filename | `nps-2010-emails.E01` |
115+
| E01 file size | 518,680 bytes (507 KB) |
116+
| E01 MD5 | `98e52ff847a440df3ba08261a3eea0f8` |
117+
| Format | EnCase 6, deflate best-compression |
118+
| Media size | 10,485,760 bytes (10 MiB) |
119+
| Sectors/chunk | 64 |
120+
| Content | 30 email addresses in various document formats |
121+
122+
**Full-media MD5:** `7dae50cec8163697415e69fd72387c01` — identical across libewf, Sleuth Kit, and ewf crate.
123+
72124
## How to Reproduce
73125

74126
### Download test images
@@ -111,5 +163,8 @@ cargo test --tests
111163
| Szechuan Sauce | EWF v1, 4 segments | 15.0 GiB | `bcd3aef...` | Yes |
112164
| MaxPowers | linen 5, 1 segment | 50.0 GiB | `10c1fbc...` | Yes |
113165
| PC-MUS-001 | EnCase 6, 1 segment | 238.5 GiB | `522df9d...` | Yes |
166+
| exfat1 | EnCase 6, compressed | 95 MiB | `0777ee9...` | Yes |
167+
| imageformat_mmls_1 | FTK Imager, uncompressed | 60 MiB | `8ec671e...` | Yes |
168+
| nps-2010-emails | EnCase 6, compressed | 10 MiB | `7dae50c...` | Yes |
114169

115-
The `ewf` crate produces bit-identical output to both libewf and The Sleuth Kit across all 303 GiB of tested media.
170+
The `ewf` crate produces bit-identical output to both libewf and The Sleuth Kit across all 6 images (303+ GiB of tested media). Images 4-6 are committed as test fixtures and run in CI.

0 commit comments

Comments
 (0)