Intelligent, modular Smart Home platform unifying IoT devices, rule automations and a conversational AI layer.
A modular smart‑home web platform (Angular + .NET) integrating multiple ecosystems (Fibaro, Home Assistant, IP cameras) with a rule automation engine and conversational AI.
Showcase repository (documentation + screenshots). Core source code is private; this material curates key architectural and product highlights for portfolio purposes.
Unified Smart Home platform consolidating fragmented IoT ecosystems into one secure, transparent control surface: devices + predictable automation + conversational AI + low‑latency camera streaming, all under a deny‑by‑default authorization model.
Problem (Fragmentation): Users juggle multiple vendor apps (Fibaro, Home Assistant, raw camera URLs) with duplicated device concepts and opaque automation failures. Problem (Lack of Transparency): Traditional rule engines give little insight into why a rule did or did not fire; debugging becomes guesswork. Problem (Cognitive Overhead): Turning intent ("Dim the hallway lights at sunset unless motion") into precise multi‑system actions requires expert knowledge.
Solution: A unified orchestration and automation platform:
- Single device & scene abstraction (Adapter + Orchestrator) spanning providers.
- Minute‑aligned automation loop with pluggable triggers & actions.
- Conversational AI layer translating natural language into structured, auditable intents.
- Managed FFmpeg pipeline producing short‑segment HLS for near real‑time camera viewing.
- Deny‑by‑default Resource/Action authorization to prevent accidental exposure.
- Rich observability: execution history + system timeline + provenance for AI answers.
Outcome: Faster rule iteration, reduced cognitive load, explainable automations, consistent UX, improved security posture, and a foundation for AI‑assisted expansion (suggested rules, anomaly insights).
| Pillar | User / Business Value | Key Mechanism |
|---|---|---|
| Devices Unification | One mental model across vendors | Adapter + Orchestrator abstraction |
| Automation Engine | Consistent & explainable outcomes | Minute‑aligned scheduler + trigger evaluators + action executors |
| AI Layer | Natural language → structured action | Embeddings + intent normalization + provenance surfacing |
| Security | Prevents accidental exposure | Deny‑by‑default Resource/Action authorization middleware |
| Streaming | Low friction situational awareness | Managed FFmpeg → short (2s) HLS segments + cleanup |
| Observability | Faster debugging & trust | Timeline + execution history + structured logs |
| Extensibility | Sustainable feature growth | Plug‑in triggers, actions, integrations (DI multi‑registration) |
Frontend (Angular SSR + Tailwind): Standalone feature domains (devices, cameras, rules, integrations, ai, auth). Reusable UI primitives keep styling consistent; interceptors centralize base URL, auth, XSRF.
Backend (.NET 10): Modular service composition; controllers thin → orchestrators / services; deterministic background worker (automation); AI chat & embedding layer; FFmpeg streaming manager.
Security: Explicit Resource/Action attributes enforced centrally; JWT + future MFA (TOTP implemented, WebAuthn planned). Rate limiting for AI endpoints.
Observability: Structured logs (Serilog), execution history & system timeline for correlation; provenance surfaced in AI responses.
Performance Considerations: Short HLS segments (2s) for latency; -codec copy to avoid unnecessary transcode; minute boundary alignment reduces scheduling drift.
Extensibility: New trigger/action/integration added via DI registration—no core refactor required.
Diagrams:
- Context:
diagrams/context.md - Containers:
diagrams/containers.md - Backend Components:
diagrams/components-backend.md - Frontend Components:
diagrams/components-frontend.md
- C4 Level 1 Context:
diagrams/context.md - C4 Level 2 Containers:
diagrams/containers.md - Backend Components:
diagrams/components-backend.md - Frontend Components:
diagrams/components-frontend.md - Sequence – Automation Execution:
diagrams/sequence-automation.md - Sequence – AI Command Orchestration:
diagrams/sequence-ai-command.md - Code Snippets (Security):
code-snippets/security/ - Code Snippets (Automation):
code-snippets/automation/ - Code Snippets (AI):
code-snippets/ai/ - Code Snippets (Streaming):
code-snippets/streaming/ - Code Snippets (Integrations):
code-snippets/integrations/
| Feature | Status | Notes |
|---|---|---|
| Unified Device Catalog | ✅ | Fibaro + Home Assistant abstraction (Tuya ready) |
| Rule Automation Engine | ✅ | Deterministic minute cycle + debounce |
| Conversational AI | ✅ | Intent normalization + provenance chunks |
| Camera Streaming | ✅ | Managed FFmpeg → 2s HLS segments |
| Deny-by-Default AuthZ | ✅ | Missing metadata ⇒ 403 enforced centrally |
| TOTP MFA Foundation | ✅ | Encrypted secrets, lockout strategy |
| WebAuthn | 🔜 | Planned (frontend scaffolding) |
| AI Rule Suggestions | 🔜 | Planned (embedding + pattern mining) |
| Energy Analytics | 🔜 | Planned (time‑series & widgets) |
The AI layer turns conversational intent into orchestrated actions while preserving traceability:
- Embed curated docs & capability descriptors (offline build step) → semantic index.
- User prompt → semantic retrieval (top relevant chunks) → context assembly.
- Intent normalization boundary converts free text into structured command (type + parameters + optional safety checks).
- If actionable → orchestrator executes device/scene/rule test; else returns informational answer.
- Provenance (used chunks + executed actions) returned to UI for trust & audit.
Sequence reference: diagrams/sequence-ai-command.md
Automation engine complements AI by executing scheduled / event-driven rules:
- Minute‑aligned loop reduces temporal drift and simplifies reasoning.
- Trigger evaluators isolate rule activation logic (time windows / event checks / future device state observers).
- Action executors perform idempotent operations against unified device abstraction.
- Execution history persists outcome (timestamp, success/failure, metadata) for debugging and analytics.
Sequence reference: diagrams/sequence-automation.md
Future focus: AI‑assisted rule template suggestions; counterfactual “why didn’t this run?” diagnostics.
Security is foundational—not retrofitted:
| Aspect | Approach | Value |
|---|---|---|
| Authorization | Deny‑by‑default Resource/Action attributes + middleware | Eliminates accidental open endpoints |
| Authentication | JWT (short‑lived) + refresh + MFA readiness (TOTP encrypted) | Layered defense |
| MFA Secret Protection | AES‑GCM versioned envelope | Integrity + future rotation |
| Rate Limiting (AI) | Catch & signal with Retry‑After header | Predictable client backoff |
| Secret Handling | Env variables for model keys / integration credentials | Avoids repo leakage |
| Defense in Depth | Central logging of deny reasons | Faster audit & forensics |
| Least Privilege | Granular Resource/Action taxonomy | Minimal surface exposure |
Snippet (auth middleware) planned: code-snippets/security/
Future items:
- WebAuthn second factor.
- Endpoint-level anomaly detection (excessive failed rule modifications).
- Optional signed HLS playlist URLs.
Goals: near real‑time camera visibility, stable resource usage, safe restarts.
- RTSP input → FFmpeg managed process → 2s HLS segments (ring buffer of ~10s window).
- Deletion flags (
delete_segments+append_list) keep storage minimal. -codec copyavoids heavy CPU; audio stripped (-an) when not needed to reduce bandwidth.- Wrapper waits for playlist readiness (ensures UI does not 404 initial load).
- Health: if no playlist update within threshold → restart routine (planned snippet).
Advantages:
- Lower latency vs traditional 6–8s segments.
- Stable memory/disk footprint via aggressive cleanup.
- Clear isolation: Streaming Service boundary keeps FFmpeg specifics from polluting higher layers.
Planned snippets: code-snippets/streaming/
Future focus: adaptive segment length, optional per‑camera key rotation, exploratory WebRTC path.
| Dimension | Mechanism | Benefit |
|---|---|---|
| Automation Transparency | Execution history (rule id, timestamp, action results) | Rapid root cause analysis |
| System Timeline | Unified event ledger (device state, rule fire, integration updates) | Correlated narrative of system behavior |
| AI Provenance | Returned chunk metadata + executed action list | Trust & audit of AI outputs |
| Structured Logging | Enriched context (ruleId, triggerType, duration) | Filterable diagnostics & metrics export |
| Authorization Denials | Reason codes (MissingMetadata / Denied) | Fast permission gap detection |
| Streaming Health | Playlist readiness & update timestamps | Early anomaly detection |
Future instrumentation:
- Latency histogram for rule evaluation vs execution.
- Per integration error budget (e.g., Fibaro API failure rate).
- OpenTelemetry traces (Controller → Orchestrator → Adapter).
Curated functional snapshots (all test / demo data). Each caption states purpose & value quickly.
| Screen | Image | Caption |
|---|---|---|
| Dashboard | ![]() |
Unified status & quick navigation hub. |
| Screen | Image | Caption |
|---|---|---|
| Chat | ![]() |
Natural language orchestration & insight. |
| AI Settings | ![]() |
Adjustable model parameters & grounding settings. |
| Screen | Image | Caption |
|---|---|---|
| Cameras Overview | ![]() |
Rapid situational awareness. |
| Live View | ![]() |
Low‑latency HLS playback. |
| Add Camera | ![]() |
Stream configuration & validation. |
| Screen | Image | Caption |
|---|---|---|
| System Timeline | ![]() |
Correlated events & audit trail. |
| Test Coverage | ![]() |
Quality signal on critical paths. |
Live functionality demonstrations showing key workflows:
Concise, value‑focused fragments (full context in respective snippet directories):
| Area | Problem | Solution (Snippet) | Engineering Value |
|---|---|---|---|
| Authorization | Accidental open endpoints | Deny-by-default middleware → deny-by-default-authorization.cs |
Shrinks attack surface, explicit contract |
| Automation | Timing drift & duplicates | Minute boundary loop + debounce → scheduler-loop.cs |
Determinism & auditability |
| Streaming | Stale segments & race on initial load | Managed FFmpeg startup wrapper → ffmpeg-start-wrapper.cs |
Reliability & low latency |
| AI Intent | Unstable model phrasing | Intent normalization boundary → intent-normalization.cs |
Provider portability, explainability |
| MFA Security | Secret leakage risk | AES‑GCM encryption layer → totp-encryption-service.cs |
Confidentiality & rotation readiness |
| Rate Limiting | Unbounded AI/login attempts | Sliding window limiter → rate-limiter.cs |
Predictable consumption |
Additional evidence snippets: TOTP encryption, rate limiter, intent normalization.
| Category | Location | Purpose |
|---|---|---|
| Context Diagram | diagrams/context.md |
External actors & high-level system boundary |
| Containers | diagrams/containers.md |
Backend vs frontend deployment view |
| Backend Components | diagrams/components-backend.md |
Service & module responsibilities |
| Frontend Components | diagrams/components-frontend.md |
Feature domain boundaries |
| Automation Sequence | diagrams/sequence-automation.md |
Rule evaluation & execution flow |
| AI Command Sequence | diagrams/sequence-ai-command.md |
Intent normalization & orchestration path |
| AuthZ Middleware | code-snippets/security/deny-by-default-authorization.cs |
Central enforcement contract |
| Automation Loop | code-snippets/automation/scheduler-loop.cs |
Deterministic slot scheduler |
| Streaming Wrapper | code-snippets/streaming/ffmpeg-start-wrapper.cs |
Reliable low-latency startup |
| Intent Normalizer | code-snippets/ai/intent-normalization.cs |
Stable command structuring |
| MFA Secret Encryption | code-snippets/security/totp-encryption-service.cs |
Protected TOTP seed storage |
| Rate Limiter | code-snippets/security/rate-limiter.cs |
Controlled consumption & lockout |
| Fibaro Device Control Demo | gifs/fibaro-device-control.gif |
Interactive RGBW device control demonstration |
| Home Assistant Integration Demo | gifs/home-assistant-dashboard.gif |
Unified dashboard entity control workflow |
| AI Interaction Demo | gifs/harmony-ai-interaction.gif |
Conversational AI device control showcase |
| Camera Streaming Demo | gifs/camera-live-view.gif |
Low-latency live stream activation |
| Automation Creation Demo | gifs/automation-rule-creation.gif |
Complete rule creation workflow |
| Garage Gate Control UI Demo | gifs/gate-control-ui.gif |
Garage gate toggle through platform UI |
| Garage Gate Opening IRL Demo | gifs/gate-opening-irl.gif |
Physical actuation proof (garage gate) |
| Fibaro RGBW IRL Demo | gifs/fibaro-rgbw-irl.gif |
Real hardware LED strip color change |
| Contrast | CONTRAST.md |
Key differentiators vs typical OSS |
I designed and implemented the full stack: architecture, abstractions, security hardening, AI orchestration, automation engine, and streaming pipeline. Key decisions and rationale:
| Decision | Rationale | Impact |
|---|---|---|
| Resource/Action deny-by-default model | Prevent silent exposure; machine-derivable permission map | Higher baseline security |
| Minute-aligned automation loop | Fixed time slots simplify reasoning & logs correlation | Predictable execution |
| Pluggable trigger/action interfaces | Localize change & enable domain growth | Faster feature iteration |
| Intent normalization boundary (AI) | Decouple model variability from orchestration logic | Provider agility & explainability |
| Short HLS segments (2s) + cleanup flags | Balance latency vs resource usage | Smooth camera UX |
| Embedding provenance surfaced to UI | Build user trust & auditing capability | Explainable AI |
| AES-GCM versioned secret format | Forward-compat for rotation & integrity | Reduced migration risk |
| Structured logging strategy (contextual keys) | Queryable diagnostics & future observability pipeline | Faster MTTR |
Leadership / Ownership aspects:
- Unified product narrative (business value + engineering clarity).
- Early security baked into architecture (not bolted on).
- Focus on extension seams to avoid architectural dead ends.
- Documentation-first ethos (diagrams + curated snippets) to accelerate reviewer comprehension.
| Theme | Item | Value |
|---|---|---|
| Security | WebAuthn 2FA | Stronger phishing resistance |
| AI Assist | Rule suggestion & refinement loop | Faster automation authoring |
| Analytics | Energy usage time-series & widgets | Insight → optimization |
| Notifications | Push (PWA) critical event alerts | Real-time awareness |
| Observability | OpenTelemetry tracing | Deep performance & dependency insight |
| Streaming | Adaptive segment length | Resilience on constrained networks |
| Hardware | Arduino-based Harmony devices | Native ecosystem sensors & actuators |
| Extensibility | Integration marketplace scaffold | Community-driven growth |
| Reviewer Lens | What This Demonstrates |
|---|---|
| Architect | Clear seams (triggers, actions, adapters) & security-first model |
| Backend Engineer | Predictable scheduling, structured logging, extensible authorization |
| Frontend Engineer | Feature isolation, SSR strategy, consistent UI primitives |
| AI Engineer | Intent normalization, provenance & retrieval pattern |
| DevOps / Reliability | Process supervision (FFmpeg), minimal drift automation design |
| Product / Stakeholder | Cohesive narrative: solves fragmentation with explainable intelligence |
- Scope: Full-stack architecture + security + AI orchestration + automation engine + streaming.
- Differentiators: Deny-by-default authZ, predictable automation loop, provenance-backed AI, managed FFmpeg latency control.
- Evidence: Curated code snippets, C4 diagrams, execution history & timeline UX.
- Outcome: Lower operational guesswork, safer extension surface, faster iteration, audit-friendly automations.
For a structured comparison vs typical open‑source smart‑home stacks (authorization, automation determinism, AI boundary, streaming control, observability) see: CONTRAST.md.
Essence of the delta:
- Security: Explicit Resource/Action deny‑by‑default (no silent exposure) vs permissive role checks.
- Automation: Minute-aligned loop (auditable) vs mixed cron/timer drift.
- AI: Intent normalization boundary + provenance vs opaque direct model calls.
- Streaming: Managed FFmpeg lifecycle (latency + resource hygiene) vs ad-hoc scripts.
- Observability: User-facing execution timeline vs log-only backend diagnostics.
Outcome: Lower cognitive load for reviewers, safer extension surface, easier root cause analysis, and explainable AI interactions by design.
Want a deeper walkthrough (architecture Q&A, security rationale, AI + automation live flow)? Reach out:
- LinkedIn: https://www.linkedin.com/in/patryk-czechowski-4600a91ba
- Optional: Add brief context (team size, interest focus: AI / Automation / Security / Streaming) so I can tailor the session.
Format & SLA:
- 30–40 min technical session (screenshare) + follow-up summary notes.
- Response SLA: < 24h CET (business days).
Privacy: Demo uses scrubbed sample data. No proprietary information from your side retained post-session.
- No private IPs (grep
192.168.) present. - No raw secrets / API tokens committed.
- Environment variable names only (e.g.
GEMINI_API_KEY) – no values. - TOTP secret handling described, not exposed.
- No internal hostnames / credentials.
- Code snippets trimmed & sanitized.
Unified Smart Home platform delivering explainable automation, conversational AI control, secure deny‑by‑default APIs and low‑latency camera streaming—engineered for determinism, extensibility and observability.
- ✨ Overview & Problem Statement
- 🚀 Key Capabilities
- 🏗 Architecture Summary
- 🤖 AI & Automation Engine
- 🔐 Security & Observability
- 🖼 Screenshots & Interactive Demos
- 🧪 Technical Highlights (Backend & Frontend)
- 🧭 Typical User Journeys
- 🛠 Technology Stack
- 🧑💻 My Role & Contributions
- 📂 Repository Content
- 🗺 Selected Roadmap
- 🎯 Portfolio Value (Why It Matters)
- 📜 License & Notes
- 🔄 TL;DR
Replaced by canonical structured equivalents:
- Overview → see TL;DR & Problem/Solution (sections 0 & 1)
- Key Capabilities → Feature Matrix & Pillars (sections 2 & 5)
- Architecture → Architecture Overview & Deep Dives Index (sections 3 & 4)
- AI & Automation → AI & Automation Flow (section 6)
- Security & Observability → sections 7 & 9 plus Security Checklist
- Screens Gallery → section 10
Legacy detailed prose intentionally removed to reduce redundancy and cognitive load.
Backend:
- Modular DI composition; multi-registration for extensibility.
- Deterministic scheduled automation loop (minute alignment + debounce) for predictable rule execution.
- Secure resource/action authorization layer (explicit metadata or rejection).
- Low-latency HLS generation via managed FFmpeg processes.
Frontend:
- Feature-isolated domains with clear models/services/UI layering.
- Adapter-driven integration panels (heterogeneous device types unified in UI).
- Reusable primitives (cards, dialogs, dynamic control panels) for consistent UX.
- AI chat with contextual suggestions & provenance (used chunks).
Illustrative coverage report excerpt (private source repository). Demonstrates commitment to targeted test depth around critical orchestration and security components.
- New user: Register → configure integrations → add camera → create first rule → observe execution.
- Power user: Drafts a rule in AI chat → system proposes structured template → refine & save.
- Admin: Adds new Fibaro devices → controls scenes → integrates into automation.
- Debug: Investigate failed rule → inspect automation history + system timeline → adjust parameters.
| Layer | Technologies |
|---|---|
| Frontend | Angular (SSR), TypeScript, Tailwind, RxJS |
| Backend | .NET 10, ASP.NET Core, Serilog |
| AI | HarmonyAI.SDK (embeddings + chat orchestration) |
| Streaming | FFmpeg (RTSP → HLS short segments) |
| Security | JWT, deny‑by‑default authorization middleware, MFA-ready concepts |
I architected and implemented the full stack: design of modular device abstraction, automation engine loop, AI intent resolution path, security hardening (deny‑by‑default), and low-latency streaming wrapper. Additionally: crafted diagnostic UX (timeline + execution history) and extensibility patterns (trigger evaluators, action executors, integration adapters).
Key decisions:
- Using resource/action metadata instead of ad-hoc policy strings → lower cognitive overhead.
- Minute-slot deterministic scheduling → reproducible automation behaviour.
- Short HLS segments with aggressive cleanup → latency vs resource balance.
- Embedding-driven AI context to ground device control suggestions.
High‑signal curated files & folders (source code itself is private; snippets & docs illustrate architecture, security, AI & automation design decisions).
| Path | Type | Purpose / Notes |
|---|---|---|
README.md |
Doc | Primary showcase narrative (problem → architecture → evidence) |
FEATURES.md |
Doc | Detailed functional & technical capability matrix |
CONTRAST.md |
Doc | Differentiators vs typical OSS smart‑home stacks |
ACCESSIBILITY.md |
Doc | Accessibility & contrast considerations |
PERFORMANCE.md |
Doc | Performance targets & rationale (latency, streaming) |
THREATMODEL.md |
Doc | Lightweight STRIDE-style threat considerations |
QUICKSTART.md |
Doc | (Showcase) orientation & navigation guide |
DIAGRAMS_INDEX.md |
Doc | Index of C4 + sequence diagrams |
diagrams/ |
Folder | C4 (context, containers, components) + sequence diagrams |
diagrams/context.md |
Diagram | External actors & system boundary |
diagrams/containers.md |
Diagram | Frontend vs backend deployment & services |
diagrams/components-backend.md |
Diagram | Service/module responsibilities (.NET) |
diagrams/components-frontend.md |
Diagram | Angular feature domain layout |
diagrams/sequence-automation.md |
Diagram | Rule evaluation & execution path |
diagrams/sequence-ai-command.md |
Diagram | Conversational intent → action orchestration |
code-snippets/ |
Folder | Curated illustrative backend excerpts |
code-snippets/automation/ |
Folder | Scheduler loop & rule engine logic |
code-snippets/security/ |
Folder | AuthZ middleware, TOTP encryption, rate limiting |
code-snippets/ai/ |
Folder | Intent normalization logic |
code-snippets/streaming/ |
Folder | FFmpeg startup & streaming wrapper |
code-snippets/integrations/ |
Folder | Integration adapter patterns (scaffold) |
gifs/ |
Media | Interactive demonstration animations (UI + real hardware) |
gifs/fibaro-device-control.gif |
Demo | RGBW UI device control |
gifs/home-assistant-dashboard.gif |
Demo | Home Assistant entity control |
gifs/harmony-ai-interaction.gif |
Demo | Conversational AI orchestration |
gifs/camera-live-view.gif |
Demo | Low‑latency streaming proof |
gifs/automation-rule-creation.gif |
Demo | Rule creation workflow |
gifs/gate-control-ui.gif |
Demo | Garage gate UI toggle |
gifs/gate-opening-irl.gif |
Demo | Physical garage gate actuation |
gifs/fibaro-rgbw-irl.gif |
Demo | Real RGBW strip reacting to command |
images/ |
Media | Static UI screenshots (gallery) |
images/app-dashboard.png |
Image | Unified system dashboard |
images/harmony-ai-chat-window.png |
Image | AI chat interface |
images/rules-list.png |
Image | Rules management overview |
images/system-history-timeline.png |
Image | System timeline (observability) |
(more in images/…) |
Image | Additional flows & forms |
- MFA hardening (WebAuthn UI, TOTP flows).
- AI-driven rule suggestions & refinement loops.
- Energy analytics (time-series & widgets).
- Push notifications (PWA) for critical events.
- Arduino-based Harmony devices (native ecosystem sensors).
- E2E test harness (Playwright + synthetic mocks).
- Community integration marketplace.
- Demonstrates end-to-end product thinking (UX + architecture + operations).
- Shows security-first mindset (explicit permission contracts).
- Highlights AI as an enabling layer, not a bolt-on feature.
- Provides observability patterns (timeline, structured execution logging).
- Scales via clear extension seams (triggers, executors, adapters).
Showcase only. Code snippets are illustrative; underlying private repository not included. Screens may display test data. Do not expose real credentials.
Modular Smart Home platform: integrations + rules + AI + low-latency streaming; designed for clarity, security, extensibility & diagnostic transparency.





























