Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
15692fb
Merge pull request #8 from chcavignx/main
chcavignx Apr 30, 2026
3aa6170
refactor: code structure for improved readability and maintainability
chcavignx May 26, 2026
32457e0
refactor: audio tests to use AudioRecorder and streamline TTS engine …
chcavignx May 27, 2026
10191f6
Merge branch 'test' of https://github.com/chcavignx/AI-Autonomous-Ass…
chcavignx May 27, 2026
6d72422
chore: update lock file
chcavignx May 27, 2026
2934288
chore: remove unused config path
chcavignx May 27, 2026
04e5810
chore: forgot a file and change github action
chcavignx May 27, 2026
d66e125
chore: change CI run_tests to be able to run on linux X86
chcavignx May 27, 2026
d456577
chore: fix uv sync
chcavignx May 27, 2026
f68746f
chore: fix CI github
chcavignx May 27, 2026
53036cd
chore: fix project file for CI action
chcavignx May 27, 2026
a049304
chore: update uv.lock
chcavignx May 27, 2026
ca96695
feat: optimise audio components and update test infrastructure.
chcavignx Jun 10, 2026
a0b43e9
refactor: Audio Testing and ASR Model Loading
chcavignx Jun 10, 2026
fd85d0b
refactor: Update Tests due to the Audio Engine refactoring
chcavignx Jun 10, 2026
c85c7ef
chore: Update Dependencies for Reproducible Builds
chcavignx Jun 10, 2026
45c995b
feat: Implement JSON-Based Python Logging Configuration
chcavignx Jun 10, 2026
26d0655
fix: error when initializing the log file
chcavignx Jun 10, 2026
ef3422f
feat: Standardize Test Execution and Coverage Reporting
chcavignx Jun 10, 2026
cb7ec2d
feat: Standardize Audio Test Execution and Coverage
chcavignx Jun 11, 2026
4e5a335
feat: Alias Sounddevice Import for Conciseness
chcavignx Jun 11, 2026
1fc2008
chore: No Substantive Technical Changes Identified
chcavignx Jun 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/python-uv-setup/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ inputs:
sync-flags:
description: Extra flags for uv sync (e.g., --locked --group test)
required: false
default: "--locked"
default: "--locked --all-groups"
install-system-packages:
description: Install Linux system packages required by audio / Raspberry Pi jobs
required: false
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ jobs:
- name: Install dependencies
uses: ./.github/actions/python-uv-setup
with:
enable-cache: true
install-system-packages: "true"
sync-flags: "--locked --no-default-groups --group lint"
- name: Run Ruff linter
run: uv run ruff check .
Expand All @@ -45,6 +47,7 @@ jobs:
- name: Install dependencies
uses: ./.github/actions/python-uv-setup
with:
install-system-packages: "true"
sync-flags: "--locked --no-default-groups --group test"
- name: Run core tests
run: uv run pytest -n auto -m "${{ inputs.marker }}" --cov=src --cov-report=xml tests/
Expand All @@ -68,6 +71,7 @@ jobs:
- name: Install dependencies
uses: ./.github/actions/python-uv-setup
with:
enable-cache: true
install-system-packages: "true"
sync-flags: "--locked --no-default-groups --group test --extra raspberry-pi"
- name: Run Raspberry Pi profile tests
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,7 @@ docs/_build/

# Environment variables
.env
.aider*
.agent*
# Added by code-review-graph
.code-review-graph/
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.12
rev: v0.15.14
hooks:
- id: ruff
name: ruff check --fix (commit)
Expand Down
52 changes: 36 additions & 16 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,46 +11,66 @@ paths:
tts:
engine: "piper"
model_name: "jarvis-medium.onnx"
# model_path: null # Let Python code build the path dynamically
model_path: Null # Let Python code build the path dynamically
cli_mode: false
device: "cpu"
length_scale: 1.0
noise_scale: 1.0
noise_w_scale: 1.0
normalize_audio: true
speed: 1.0
volume: 1.0
volume: 0.5

vad:
min_speech_duration_ms: 250
min_speech_duration_ms: 100
min_silence_duration_ms: 500
silence_timeout_seconds: 3
max_recording_seconds: 10
threshold: 0.5
silence_timeout_seconds: 1
max_recording_seconds: 15
threshold: 0.6
sample_rate: 44100

# Automatic Speech Recognition (ASR) configuration
asr:
engine: "faster-whisper" # "whisper", "faster-whisper"
model_size: "tiny"
faster_model_size: "small"
language: "en"
translate: false
transformers: false
transformers: true
transformers_engine: "huggingface" # "huggingface" or "onnxruntime"
device: "cpu" # Pi5 : CPU (or "hailo")
compute_type: "int8"
skip_native_teardown: true # Avoid native teardown segfaults on some ARM/PortAudio stacks
# download_root: null # Let Python code build the path dynamically
download_root: null # Let Python code build the path dynamically
skip_native_teardown: false # Avoid native teardown segfaults on some ARM/PortAudio stacks
store_audio: true
store_audio_path: ".tmp/asr.wav"


wake:
wake_word: "hey_jarvis"
model_name: "hey_jarvis_v0.1"
model_name: "embedding_model"
model_path: null
inference_framework: "onnx"
threshold: 0.5
threshold: 0.4
cooldown_seconds: 2.0
noise_suppression: true
download_root: null
vad_threshold: 0.6

audio:
input_sample_rate: 44100
input_sample_rate: 48000
input_chunk_ms: 30 # taille des chunks audio en ms
input_chunk_size: 300
input_device_index: 2 # No input-capable PyAudio device detected on this machine; use system default when a mic is attached
volume: 0.8 # half as loud
output_device_index: 0 # PyAudio output device: USB PnP Audio Device: Audio (hw:3,0)
input_chunk_size: 500
input_device_index: 0 # use system default when a mic is attached
input_device_name: 'USB ENC Audio Device' # Optional name of input device to select (overrides index if found)
volume: 0.5 # half as loud
output_device_index: 1 # default to USB PnP Audio Device: Audio (hw:3,0)
output_sample_rate: 44100
output_chunk_ms: 30 # taille des chunks audio en ms
output_chunk_size: 500
output_device_name: 'USB PnP Audio Device' # Optional name of output device to select (overrides index if found)

# Platform-specific configuration
platform:
cpu_cores: 2 # Limit CPU cores for multiprocessing on Pi5
pi: false # Automatically detect Raspberry Pi and apply tuning
8 changes: 4 additions & 4 deletions docs/DEV_PROCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,10 @@ Here is a prioritization proposed for the context: Raspberry Pi 5, Hailo-8L, NVM
### Sequenced Guides for Different Modules

- **Audio Modules**
1. [Offline Speech Recognition (STT)](STT_offline.md)
2. [Offline Text-to-Speech (TTS)](TTS_offline.md)
3. [Voice stack and VAD models](STS_VAD_models.md)
4. [USB microphone and speaker test](audio_usb_test.md)
1. [USB microphone and speaker test](audio_usb_test.md)
2. [Offline Speech Recognition (STT)](STT_offline.md)
3. [Offline Text-to-Speech (TTS)](TTS_offline.md)
4. [Voice stack and VAD models](STS_VAD_models.md)
5. [Offline Speech-to-Speech demo](../examples/VAD/voice_agent_offline.md)
Demo application that listens for a wake word, transcribes the next utterance, generates a keyword response, and speaks it back (`examples/VAD/voice_agent_offline.py`)

Expand Down
5 changes: 2 additions & 3 deletions docs/STS_VAD_models.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Voice Activity Detection (VAD) is a critical component in modern speech processi

## Most Appropriate Choice for AI Autonomous Agents: Silero VAD

### Why Silero VAD is Optimal:
### Why Silero VAD is Optimal

1. **Ultra-lightweight**: 1-2MB model size with <1ms inference time per chunk
2. **Multilingual capability**: Trained on 100+ languages with robust performance
Expand Down Expand Up @@ -110,7 +110,6 @@ Voice Activity Detection (VAD) is a critical component in modern speech processi
| **Vosk** | ⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ✅ Excellent | ⭐⭐⭐ | ✅ Excellent | Easy |
| **SpeechRecognition** | ⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ✅ Excellent | ⭐⭐ | ✅ Excellent | Very Easy |


## STS Architecture Patterns for Autonomous Agents - Recommended Combinations

**Key Finding**: The optimal solution combines models for the best balance of accuracy, performance, and resource efficiency on Raspberry Pi 5 hardware.
Expand Down Expand Up @@ -204,7 +203,7 @@ asr.model_size: tiny
asr.device: cpu
asr.compute_type: int8
tts.engine: piper
tts.model_name: en_US-hfc_female-medium.onnx
tts.model_name: jarvis-medium.onnx
wake.wake_word: hey_jarvis
```

Expand Down
2 changes: 1 addition & 1 deletion docs/STT_offline.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Default ASR settings come from `src/utils/config.py`:
Install the runtime pieces used by the current ASR engine:

```bash
pip install pyaudio torch scipy silero-vad faster-whisper
pip install sounddevice torch scipy silero-vad faster-whisper
```

Install OpenAI Whisper only if you want to switch the config to `engine: whisper`:
Expand Down
16 changes: 3 additions & 13 deletions docs/TTS_offline.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ The engine uses the `piper-tts` Python package for the default mode and supports
Default TTS settings come from `src/utils/config.py`:

- `engine = piper`
- `model_name = en_US-hfc_female-medium.onnx`
- `cli_mode = false`
- `model_name = jarvis-medium.onnx`
- `model_path = Null`
- `speed = 1.0`
- `volume = 0.5`(range 0.0 to 1.0)
- `output_sample_rate = 22050`(common for TTS models, but can be adjusted based on the model's requirements and playback capabilities)
Expand All @@ -109,25 +109,15 @@ The implementation is non-blocking by default, so the assistant can keep listeni

#### Piper Python API

- Used when `cli_mode = false`
- Requires the `piper-tts` Python package
- Loads the voice model directly from the configured model path

#### Piper CLI

- Used when `cli_mode = true`
- Searches for a `piper` binary in common locations:
- `~/.local/bin/piper`
- `/usr/local/bin/piper`
- `/usr/bin/piper`
- Useful when you prefer the standalone Piper binary

### Installation

Install the main dependencies used by the current TTS engine:

```bash
pip install piper-tts pyaudio
pip install piper-tts sounddevice
```

If you want to use the Piper CLI mode, also install the Piper binary from the official project.
Expand Down
145 changes: 144 additions & 1 deletion examples/AUDIO_TESTS_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ These tests follow a **simple-first, build-progressively** approach:
2. **Stream Control** - Can we open/close audio streams?
3. **Playback** - Can we play audio through output?
4. **Recording** - Can we capture audio from input?
5. **Recorder Standalone** - Can the simplified AudioRecorder capture and format chunks?
6. **ASR Integration** - Can the high-level ASREngine capture and transcribe chunks?
7. **ASR with TTS** - Can ASREngine transcribe TTS audio output?
8. **ASR Recording Validation** - Can we transcribe microphone recordings?
9. **Wake Word Standalone** - Can the WakeWordDetector load models and capture audio in background?
10. **VAD Standalone** - Can the deep learning Silero VAD engine detect speech segments and handle resampling?
11. **TTS Lifecycle & Utils** - Can we manage non-blocking TTS, interruptions, and utility conversions?

Each test is **self-contained and can run independently** on any system with audio hardware.

Expand Down Expand Up @@ -86,6 +93,7 @@ python examples/test_stream_open_close.py
- Generate a sine wave (440 Hz)
- Send it to output stream
- Also tests silence playback (zeros)
- **New:** Validates `AudioPlayer.play_file()` with real WAV files.

**Run:**

Expand All @@ -98,6 +106,7 @@ python examples/test_playback.py
- Stream opens successfully
- Data writes without errors
- Stream responds to stop command
- File-based playback routes correctly through backend

**Skips if:** No output device available.

Expand Down Expand Up @@ -135,13 +144,147 @@ python examples/test_recording.py

---

### 5. Recorder Standalone (`test_recorder_standalone.py`)

**Purpose:** Validate the simplified `AudioRecorder` utility class.

**What it tests:**

- Can the recorder start/stop cleanly?
- Does `read_numpy()` return valid 16-bit PCM?
- Does `read_float()` return normalized [-1, 1] data?

**Run:**

```bash
python examples/test_recorder_standalone.py
```

---

### 6. ASR Engine Integration (`test_asr_integration.py`)

**Purpose:** Verify the full ASR pipeline using the production `ASREngine`.

**What it tests:**

- Loads the ASR model (Whisper/Faster-Whisper)
- Validates that the engine thread captures audio chunks
- Checks that the VAD (Voice Activity Detection) system is initialized
- Verifies the callback system for transcriptions

**Run:**

```bash
python examples/test_asr_integration.py
```

**Skips if:** No input device available.

---

### 7. ASR with TTS (`test_asr_with_tts.py`)

**Purpose:** Verify that the high-level `ASREngine` can accurately transcribe speech generated dynamically by the `TTSEngine` (loopback integration).

**What it tests:**

- Generates voice samples via PIPER `TTSEngine`
- Feeds synthesized WAV bytes directly to the Whisper `ASREngine`
- Asserts that original text matches transcribed text within word-error tolerance thresholds

**Run:**

```bash
python examples/test_asr_with_tts.py
```

---

### 8. ASR Recording Validation (`test_asr_recording_validation.py`)

**Purpose:** Validate dynamic user recording capture and transcription pipeline.

**What it tests:**

- Records 3 seconds of active microphone input
- Streams recording frames to `ASREngine`
- Returns real-time speech transcription to verify complete mic-to-text pipeline

**Run:**

```bash
python examples/test_asr_recording_validation.py
```

**Skips if:** No input device available.

---

### 9. Wake Word Standalone (`test_wake_word_standalone.py`)

**Purpose:** Validate the `WakeWordDetector` engine in isolation.

**What it tests:**

- Loads openWakeWord models
- Spawns background capture and detection threads
- Verifies callback triggers upon detection

**Run:**

```bash
python examples/test_wake_word_standalone.py
```

---

### 10. VAD Standalone Flow (`test_vad_standalone.py`)

**Purpose:** Validate voice activity detection state machines and audio sample rate converters.

**What it tests:**

- Initializes standalone `VADEngine` with deep-learning Silero VAD
- Loads a real speech sample (`data/test.wav`) to verify positive speech identification (`is_speech_detected() -> True`) and correct voice active timestamps/segments (`get_speech_segments()`)
- Tests negative identification against synthetic silence (`is_speech_detected() -> False`)
- Tests native audio high-quality downsampling/upsampling logic (`resample_audio()`) between 16000Hz and 22050Hz

**Run:**

```bash
python examples/test_vad_standalone.py
```

---

### 11. TTS Lifecycle & Utils (`test_tts_lifecycle_and_utils.py`)

**Purpose:** Validate advanced non-blocking TTSEngine APIs and low-level utility operations.

**What it tests:**

- Initializes and loads PIPER `TTSEngine` with active local models
- Tests asynchronous dynamic speech generation (`speak(..., blocking=False)`)
- Tests state queries (`is_speaking`) and queue interruption/abort triggers (`interrupt()`)
- Tests synchronization blocks (`wait()`) and teardowns (`unload()`)
- Asserts correctness of untested `audio_utils.py` functions: `convert_to_float32()`, `convert_to_int16()`, `resolve_device_index()`, `suppress_pa_stderr()`, and `install_alsa_error_handler()`

**Run:**

```bash
python examples/test_tts_lifecycle_and_utils.py
```

---

## Run All Tests

```bash
python examples/run_all_audio_tests.py
```

Runs all 4 tests in sequence with a summary report.
Runs all 11 tests in sequence with a summary report.

---

Expand Down
Loading
Loading