diff --git a/.gitignore b/.gitignore index 55355c3..965594f 100644 --- a/.gitignore +++ b/.gitignore @@ -22,3 +22,4 @@ makefile.* # Analyzer logs written by --output json or a follow session. logs/ +/*.json diff --git a/CHANGELOG.md b/CHANGELOG.md index de957e5..ecaa838 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,16 @@ All notable changes to this project are documented here. ## [Unreleased] +## [0.1.2] - 2026-09-14 + +### Fixed + +- `--output json` now writes one valid JSON array. A new scan replaces stale output, while follow + mode appends new detections without producing concatenated top-level JSON objects. +- Correlation detections are now included in JSON and combined output instead of appearing only in + the console path. +- The crate and binary version now match the published release tag. + ### Added - `CONTRIBUTING.md`, `docs/architecture.md`, and `docs/rules-schema.md`. The module map, the diff --git a/Cargo.lock b/Cargo.lock index b8158bc..f4e6c4d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -624,7 +624,7 @@ checksum = "2b15c43186be67a4fd63bee50d0303afffcef381492ebe2c5d87f324e1b8815c" [[package]] name = "rust_hound" -version = "0.1.0" +version = "0.1.2" dependencies = [ "anyhow", "chrono", diff --git a/Cargo.toml b/Cargo.toml index 76707f4..1c70e91 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "rust_hound" -version = "0.1.0" +version = "0.1.2" edition = "2021" rust-version = "1.85" diff --git a/README.md b/README.md index 6b18f18..9f8fefd 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ The verified sample run emits eight detections with severity and source-line con cargo run --locked -- --init-config ~~~ -For JSON output, use `--output json`; for both console and JSON, use `--output both`. `--follow` monitors appended content. `--dir PATH` scans regular `.log` files in a directory. +For JSON output, use `--output json`; for both console and JSON, use `--output both`. The output file is one JSON array and a new non-follow scan replaces stale results instead of duplicating them. In `--follow` mode, new detections are appended while the file remains a valid JSON document. `--dir PATH` scans regular `.log` files in a directory. ## Installation @@ -99,7 +99,7 @@ cargo test --locked cargo +1.85.0 check --locked --all-targets ~~~ -The current local run passes 12 library tests, no duplicate binary test suite, and the doctest target. The sample CLI invocation above is a real file-processing smoke test, not a benchmark. +The current local run passes 15 library tests, no duplicate binary test suite, and the doctest target. The sample CLI invocation above is a real file-processing smoke test, not a benchmark. ## Scope and limitations diff --git a/sample.json b/sample.json deleted file mode 100644 index faac6f8..0000000 --- a/sample.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "timestamp": "2025-06-30T18:55:43.332743623+00:00", - "severity": "ERROR", - "rule_name": "ERROR", - "file_path": "sample.log", - "line_number": 3, - "matched_line": "ERROR: Something went wrong.", - "pattern": "ERROR" -} -{ - "timestamp": "2025-06-30T18:55:43.332903427+00:00", - "severity": "WARNING", - "rule_name": "WARN", - "file_path": "sample.log", - "line_number": 4, - "matched_line": "WARNING: This is a warning.", - "pattern": "WARN" -} -{ - "timestamp": "2025-06-30T18:55:43.333021917+00:00", - "severity": "ERROR", - "rule_name": "ERROR", - "file_path": "sample.log", - "line_number": 5, - "matched_line": "2024-06-30 15:23:45 ERROR Database connection failed: timeout", - "pattern": "ERROR" -} -{ - "timestamp": "2025-07-04T13:43:21.121860833+00:00", - "severity": "critical", - "rule_name": "oom_kill", - "file_path": "sample.log", - "line_number": 1, - "matched_line": "Jul 1 10:00:01 hostname kernel: Out of memory: Kill process 1234 (my_app) score 999 or sacrifice child", - "pattern": "oom_kill" -} -{ - "timestamp": "2025-07-04T13:43:21.121988109+00:00", - "severity": "high", - "rule_name": "service_start_failure", - "file_path": "sample.log", - "line_number": 2, - "matched_line": "Jul 1 10:00:05 hostname systemd[1]: Failed to start Apache HTTP Server.", - "pattern": "service_start_failure" -} -{ - "timestamp": "2025-07-04T13:43:21.121999408+00:00", - "severity": "high", - "rule_name": "authentication_failure", - "file_path": "sample.log", - "line_number": 3, - "matched_line": "Jul 1 10:00:10 hostname sshd[5678]: error: PAM: authentication failure for user root from 192.168.1.10", - "pattern": "authentication_failure" -} -{ - "timestamp": "2025-07-04T13:43:21.122177272+00:00", - "severity": "high", - "rule_name": "service_start_failure", - "file_path": "sample.log", - "line_number": 6, - "matched_line": "Jul 1 10:00:25 hostname systemd[1]: Unit user@1000.service entered failed state.", - "pattern": "service_start_failure" -} -{ - "timestamp": "2025-07-04T13:43:21.122191932+00:00", - "severity": "high", - "rule_name": "authentication_failure", - "file_path": "sample.log", - "line_number": 7, - "matched_line": "Jul 1 10:00:30 hostname sudo: pam_unix(sudo:auth): authentication failure; logname=user uid=1000 euid=0 tty=/dev/pts/0 ruser=user rhost= user=user", - "pattern": "authentication_failure" -} -{ - "timestamp": "2025-07-04T13:43:21.122198945+00:00", - "severity": "high", - "rule_name": "filesystem_mount_failure", - "file_path": "sample.log", - "line_number": 8, - "matched_line": "Jul 1 10:00:35 hostname systemd[1]: Failed to mount /home/user/data.", - "pattern": "filesystem_mount_failure" -} -{ - "timestamp": "2025-07-04T13:43:21.122222458+00:00", - "severity": "critical", - "rule_name": "cpu_soft_lockup", - "file_path": "sample.log", - "line_number": 9, - "matched_line": "Jul 1 10:00:40 hostname kernel: NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/1:1:12345]", - "pattern": "cpu_soft_lockup" -} -{ - "timestamp": "2025-07-04T13:43:21.122336611+00:00", - "severity": "high", - "rule_name": "service_start_failure", - "file_path": "sample.log", - "line_number": 10, - "matched_line": "Jul 1 10:00:45 hostname systemd[1]: Dependency failed for Local File Systems.", - "pattern": "service_start_failure" -} diff --git a/src/output/json_writer.rs b/src/output/json_writer.rs index ebf5d32..98f10d0 100644 --- a/src/output/json_writer.rs +++ b/src/output/json_writer.rs @@ -1,8 +1,10 @@ -use serde::Serialize; -use std::fs::File; -use std::io::{self, Write}; +use serde::{Deserialize, Serialize}; +use std::io; +use std::path::Path; -#[derive(Serialize, Debug)] +use super::Detection; + +#[derive(Serialize, Deserialize, Debug)] pub struct AnomalyDetection { pub timestamp: String, pub severity: String, @@ -13,8 +15,113 @@ pub struct AnomalyDetection { pub pattern: String, } -pub fn write_json_output(detection: &AnomalyDetection, output_file: &mut File) -> io::Result<()> { - let json_string = serde_json::to_string_pretty(detection)?; - writeln!(output_file, "{json_string}")?; - Ok(()) +impl AnomalyDetection { + pub fn from_detection(detection: &Detection) -> Self { + Self { + timestamp: chrono::Local::now().to_rfc3339(), + severity: detection.severity.as_str().to_owned(), + rule_name: detection.pattern_name.clone(), + file_path: detection.file_path.clone(), + line_number: detection.line_number, + matched_line: detection.matched_line.clone(), + pattern: detection.pattern_name.clone(), + } + } +} + +pub fn write_json_output( + output_path: &Path, + detections: Vec, + append: bool, +) -> io::Result<()> { + let mut output = if append && output_path.exists() { + let bytes = std::fs::read(output_path)?; + serde_json::from_slice::>(&bytes)? + } else { + Vec::new() + }; + output.extend(detections); + + let mut bytes = serde_json::to_vec_pretty(&output)?; + bytes.push(b'\n'); + std::fs::write(output_path, bytes) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::time::{SystemTime, UNIX_EPOCH}; + + fn detection(rule_name: &str) -> AnomalyDetection { + AnomalyDetection { + timestamp: "2026-09-14T00:00:00Z".to_owned(), + severity: "warning".to_owned(), + rule_name: rule_name.to_owned(), + file_path: "sample.log".to_owned(), + line_number: 1, + matched_line: "WARN sample".to_owned(), + pattern: rule_name.to_owned(), + } + } + + fn output_path() -> std::path::PathBuf { + let nonce = SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("system clock must be after epoch") + .as_nanos(); + std::env::temp_dir().join(format!( + "rusthound-json-writer-{}-{nonce}.json", + std::process::id() + )) + } + + #[test] + fn writes_one_valid_json_array() { + let path = output_path(); + write_json_output(&path, vec![detection("first"), detection("second")], false) + .expect("JSON output should be written"); + + let value: serde_json::Value = + serde_json::from_slice(&std::fs::read(&path).expect("JSON output should be readable")) + .expect("output should be one valid JSON document"); + assert_eq!(value.as_array().map(Vec::len), Some(2)); + + std::fs::remove_file(path).expect("temporary output should be removable"); + } + + #[test] + fn converts_console_detections_without_losing_severity() { + let source = Detection { + severity: super::super::Severity::Critical, + file_path: "auth.log".to_owned(), + line_number: 42, + pattern_name: "Potential Brute-Force Attack".to_owned(), + matched_line: "login accepted".to_owned(), + }; + + let output = AnomalyDetection::from_detection(&source); + assert_eq!(output.severity, "critical"); + assert_eq!(output.rule_name, source.pattern_name); + assert_eq!(output.line_number, 42); + } + + #[test] + fn replaces_on_new_scan_and_appends_in_follow_mode() { + let path = output_path(); + write_json_output(&path, vec![detection("old")], false) + .expect("initial JSON output should be written"); + write_json_output(&path, vec![detection("replacement")], false) + .expect("a new scan should replace stale output"); + write_json_output(&path, vec![detection("follow")], true) + .expect("follow mode should append to the existing array"); + + let output: Vec = + serde_json::from_slice(&std::fs::read(&path).expect("JSON output should be readable")) + .expect("output should remain a valid JSON array"); + assert_eq!(output.len(), 2); + assert_eq!(output[0].rule_name, "replacement"); + assert_eq!(output[1].rule_name, "follow"); + + std::fs::remove_file(path).expect("temporary output should be removable"); + } } diff --git a/src/output/mod.rs b/src/output/mod.rs index 111da59..de7a903 100644 --- a/src/output/mod.rs +++ b/src/output/mod.rs @@ -33,6 +33,16 @@ impl From<&str> for Severity { } impl Severity { + pub fn as_str(&self) -> &'static str { + match self { + Severity::Critical => "critical", + Severity::High => "high", + Severity::Warning => "warning", + Severity::Error => "error", + Severity::Info => "info", + } + } + pub fn rank(&self) -> u8 { match self { Severity::Critical => 5, diff --git a/src/watcher/log_reader.rs b/src/watcher/log_reader.rs index 03b9bd0..7e2c86b 100644 --- a/src/watcher/log_reader.rs +++ b/src/watcher/log_reader.rs @@ -59,22 +59,12 @@ pub async fn read_file_from_offset( mut offset: u64, mut current_line_number: usize, ) -> anyhow::Result<(u64, usize, Vec)> { + let starting_offset = offset; let mut file = File::open(file_path).await?; file.seek(SeekFrom::Start(offset)).await?; let reader = BufReader::new(file); let mut lines = reader.lines(); - let mut json_output_file: Option = None; - if output_format == "json" || output_format == "both" { - let output_path = file_path.with_extension("json"); - json_output_file = Some( - std::fs::OpenOptions::new() - .create(true) - .append(true) - .open(&output_path)?, - ); - } - if offset == 0 && (output_format == "json" || output_format == "both") { println!( "Writing JSON output to: {}", @@ -85,6 +75,7 @@ pub async fn read_file_from_offset( println!("Reading file: {}", file_path.display()); let mut detections: Vec = Vec::new(); + let mut json_detections = Vec::new(); while let Some(line) = lines.next_line().await? { current_line_number += 1; @@ -112,9 +103,7 @@ pub async fn read_file_from_offset( matched_line: line.clone(), pattern: pattern_name.to_string(), }; - if let Some(output_file) = json_output_file.as_mut() { - crate::output::json_writer::write_json_output(&json_detection, output_file)?; - } + json_detections.push(json_detection); } if let Some(tracker) = &mut scan_state.frequency_tracker { @@ -142,12 +131,7 @@ pub async fn read_file_from_offset( matched_line: line.clone(), pattern: pattern_name.to_string(), }; - if let Some(output_file) = json_output_file.as_mut() { - crate::output::json_writer::write_json_output( - &json_detection, - output_file, - )?; - } + json_detections.push(json_detection); } } } @@ -155,6 +139,13 @@ pub async fn read_file_from_offset( if let Some(correlated_detection) = scan_state.correlation_engine.add_detection(detection) { + if output_format == "json" || output_format == "both" { + json_detections.push( + crate::output::json_writer::AnomalyDetection::from_detection( + &correlated_detection, + ), + ); + } if output_format == "console" || output_format == "both" { detections.push(correlated_detection); } @@ -162,6 +153,14 @@ pub async fn read_file_from_offset( } } + if output_format == "json" || output_format == "both" { + crate::output::json_writer::write_json_output( + &file_path.with_extension("json"), + json_detections, + starting_offset > 0, + )?; + } + Ok((offset, current_line_number, detections)) }