This repository serves as an index of experimental projects, evaluations, proof-of-concepts, templates, patterns, and exploratory ideas related to AI/LLM development and workflows.
This collection brings together various experimental repositories exploring AI agent workflows, LLM capabilities, evaluation frameworks, and development patterns. These repositories represent hands-on experiments, proof-of-concepts, benchmarking efforts, and reusable templates for AI-driven development.
| Repository | Category | Key Finding |
|---|---|---|
| Whisper Fine-Tune Accuracy Eval | Speech | Smaller models improve with fine-tuning; larger models degrade unless handling code-switching |
| One-Shot Transcription Microphone Eval | Speech | Environment matters more than equipment cost for STT accuracy |
| Transcription Cleanup Eval | Speech | Compares cloud models on single-step transcription + cleanup |
| Whisper WPM Background Noise Eval | Speech | Speaking pace and background noise impact on Whisper accuracy |
| Long Form Audio Eval | Speech | Long-form audio transcription evaluation |
| Local ASR STT Benchmark | Speech | Local ASR/STT benchmarking |
| Hebrew Image Generation Eval | Image | Hebrew text rendering in AI image generation |
| Bias Censorship Eval Tests | LLM | Testing for bias and censorship in LLMs |
| Repository | Category | Description |
|---|---|---|
| Voice Cloning Difference Test | Speech | How training data duration affects voice cloning quality |
| Text Cleanup Fine-Tuning Set | Speech | Dataset for training AI to clean up STT transcripts |
| Voice Cleanup Prompt Experiment | Speech | Comparing OpenAI vs Gemini for transcript cleanup |
| Impact Bond Policy Simulator | Multi-Agent | Simulating stakeholder reactions to policy proposals |
| Peace In The Middle East | Multi-Agent | AI simulation of geopolitical dialogue |
| Weird AI Experiment Ideator | Multi-Agent | Blind multi-pass review for generating experiment ideas |
| LLM Long Codegen Test | LLM | Testing long-form code generation |
| Single Shot Brevity Training | LLM | Training for concise responses |
| Repository | Link | Description | Date |
|---|---|---|---|
| Agent Handover Demo |  |
Demonstration of agent handover patterns | 2025 |
| Agent Network Expander Template | |
Template for expanding agent networks | 2025 |
| Agent Task Repo Pattern With MCP | |
Repository pattern for agent tasks using MCP | 2025 |
| AI Agent UN | |
AI agent unified namespace | 2025 |
| AI Agent Workspace Spec | |
Agent workspace specification | Mar 2025 |
| Weird AI Experiment Ideator | |
CrewAI multi-agent system using blind multi-pass review to generate creative AI experiment ideas | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| AI Assistant Template | |
Template for AI assistant development | 2025 |
| AI Dev Prompts Example | |
Example development prompts for AI | 2025 |
| AI Development Template | |
General AI development template | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Bias Censorship Eval Tests | |
Evaluation tests for bias and censorship | 2025 |
| LLM Evaluation Prompts | |
Prompts for evaluating LLMs | 2025 |
| Assistant Self Ideation | |
Demo of AI "self ideation" in practice | Feb 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| LLM Experiment Notebook | |
Notebook of LLM experiments | 2025 |
| LLM Long Codegen Test | |
Testing long-form code generation | 2025 |
| LLM Max Token Length | |
Maximum token length exploration | Feb 2025 |
| Single Shot Brevity Training | |
Training for concise single-shot responses | 2025 |
| One Prompt AI Book | |
Experiment in generating content from single prompts | 2025 |
| Long AI Prompting Experiment | |
Testing experiments with extended prompts | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Single Shot Brevity Training |  |
Brevity training interface | 2025 |
| LLM Long Code Generation Experiment | |
Long-form code generation experiment | 2025 |
| Max Output Tokens Analysis | |
Maximum output tokens analysis | Feb 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Local ASR STT Benchmark | |
Local ASR and STT benchmarking | 2025 |
| Long Form Audio Eval | |
Evaluation of long-form audio transcription | 2025 |
| Personal STT Benchmarking | |
Personal speech-to-text benchmarking | 2025 |
| STT Voice Note Evaluation | |
Evaluation of STT for voice notes | 2025 |
| Whisper WPM Background Noise Eval | |
Evaluating how speaking pace and background noise affect Whisper ASR accuracy | 2025 |
| Whisper Fine-Tune Accuracy Eval | |
GUI tool for comparing fine-tuned vs original Whisper models using WER metrics with whisper.cpp/Vulkan acceleration | 2025 |
| One-Shot Transcription Microphone Eval | |
Microphone benchmarking for STT—found environment matters more than equipment cost across 10 mics and 15 samples | 2025 |
| Transcription Cleanup Eval | |
Evaluates cloud audio models (GPT-4o, Gemini, Voxtral, Qwen) on single-step transcription with cleanup | Dec 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Single Podcast ASR Eval | |
Single podcast ASR evaluation | 2025 |
| STT Comparison | |
Speech-to-text comparison tool | 2025 |
| Local STT Eval One Sample | |
Local STT evaluation with single samples | 2025 |
| Whisper Fine-Tune Eval | |
Interactive evaluation of fine-tuned Whisper models | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Podcast ASR Evaluation |  |
Dataset for podcast ASR evaluation | 2025 |
| Whisper Fine-Tune One Shot Eval | |
WER and accuracy evaluation comparing fine-tuned Whisper (Tiny, Base, Small, Medium) vs stock models on 1 hour of audio, inference on Modal A100 | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Microphone Audio Samples | |
Collection of microphone audio samples | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Crying Baby Audio Scrub | |
Audio processing for baby noise removal | 2025 |
| Audio Context Pipeline Model | |
Notes and model for audio context pipeline | Apr 2025 |
| Voice Cleanup Prompt Experiment | |
Comparing OpenAI (Whisper+GPT-4) vs Gemini for transcript cleanup | 2025 |
| Voice Cloning Difference Test | |
Experiment testing how training data duration (1/3/5 min) affects one-shot voice cloning quality | 2025 |
| Text Cleanup Fine-Tuning Set | |
Dataset and tooling for training AI to automatically clean up STT transcripts | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Hebrew Image Generation Eval | |
Evaluation of AI image generation models for Hebrew text rendering | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Impact Bond Policy Simulator | |
CrewAI multi-agent framework simulating stakeholder reactions to Pay-for-Success impact bond proposals | 2025 |
| Peace In The Middle East | |
Experimental multi-agent AI system simulating geopolitical dialogue with state and non-state actors | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| OSINT Missile Intelligence Agent | |
OSINT-focused intelligence agent | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| GHG EBITDA Correlations | |
Analysis of greenhouse gas and EBITDA correlations | 2025 |
| Repository | Link | Description | Date |
|---|---|---|---|
| Test Markdown Docs | |
Test repository for markdown documentation | 2025 |
| Test System Prompts | |
Test repository for system prompts | 2025 |
| Index | Link | Description |
|---|---|---|
| Speech & ASR Evaluations | |
Comprehensive index of speech recognition and ASR evaluation studies |
Note: This is a focused index covering experimental AI/LLM development projects. For a higher-level collection of all repository indexes and other projects, see the GitHub Master Index.
Daniel Rosehill Contact: public@danielrosehill.com Website: danielrosehill.com