Skip to content

Repository files navigation

Autonomous Website Monitor & Analyst

A fully autonomous website monitoring and analysis system built on the Cloudflare platform, featuring real-time alerts and AI-powered content change detection.

🏗️ Architecture Overview

This application demonstrates the integration of four key Cloudflare services:

┌─────────────────────────────────────────────────────────────────┐
│                    CLOUDFLARE PLATFORM                          │
│                                                                 │
│  ┌──────────────┐         ┌─────────────────┐                 │
│  │   Pages      │◄────────│  Realtime       │                 │
│  │  (Frontend)  │ WebSocket│  (Bi-directional│                │
│  └──────┬───────┘         │   Communication)│                 │
│         │                 └────────┬────────┘                 │
│         │                          │                           │
│         └──────────┬───────────────┘                           │
│                    │                                           │
│                    ▼                                           │
│         ┌─────────────────────┐                               │
│         │   Workers           │                               │
│         │  (Agents SDK)       │◄──── Orchestration            │
│         │  "The Brain"        │                               │
│         └──────────┬──────────┘                               │
│                    │                                           │
│         ┌──────────┴──────────┐                               │
│         │                     │                               │
│         ▼                     ▼                               │
│  ┌─────────────┐      ┌─────────────┐                        │
│  │  Durable    │      │  Workers AI │                        │
│  │  Objects    │◄────►│  (Llama 3.3)│                        │
│  │  (State +   │      │             │                        │
│  │   Alarms)   │      └─────────────┘                        │
│  └─────────────┘                                              │
│      │                                                        │
│      │ Autonomous                                             │
│      │ Monitoring                                             │
│      ▼                                                        │
│  [Internet]                                                   │
│   Websites                                                    │
└─────────────────────────────────────────────────────────────────┘

Component Breakdown

1. Frontend - Cloudflare Pages

  • Simple chat-like UI for user interaction
  • Connects to backend via Cloudflare Realtime WebSocket
  • Receives proactive alerts without polling
  • Location: public/index.html

2. Orchestration - Workers + Agents SDK

  • Main Worker acts as the "brain" using Agents SDK
  • Interprets user commands from the chat interface
  • Routes requests to appropriate Durable Objects
  • Coordinates between all services
  • Location: src/index.ts

3. State & Autonomy - Durable Objects

  • Each monitored website gets its own DO instance
  • Stores persistent state (URL, content hash, config)
  • Uses Alarms for autonomous scheduled checks (every 10 minutes)
  • Performs website fetching and change detection
  • Location: src/durable-objects/WebsiteMonitor.ts

4. Intelligence - Workers AI (Llama 3.3)

  • Analyzes HTML content changes semantically
  • Summarizes differences in natural language
  • Focuses on meaningful content changes, not code changes
  • Integrated directly into the Durable Object

🔄 Application Flow

1. User Initiates Monitoring

User → Chat UI → "monitor example.com"
         ↓
    Realtime WebSocket
         ↓
   Agents SDK Worker (interprets command)
         ↓
   Creates/Gets Durable Object for "example.com"
         ↓
   DO stores URL & sets alarm (10 min)
         ↓
   Confirmation sent back to user via Realtime

2. Autonomous Monitoring (Every 10 minutes)

Alarm fires in Durable Object
         ↓
   Fetch website (example.com)
         ↓
   Check HTTP status
         ├─ If not 200 → Send "site down" alert
         └─ If 200 → Continue
                ↓
         Calculate content hash
                ↓
         Compare with previous hash
         ├─ If same → No action
         └─ If different → Continue
                   ↓
            Call Llama 3.3 via Workers AI
                   ↓
            Analyze old vs new HTML
                   ↓
            Get semantic summary
                   ↓
            Send alert via Realtime to user
                   ↓
            User sees proactive notification in chat

🚀 Setup & Deployment

Prerequisites

Installation

  1. Clone and install dependencies:
npm install
  1. Configure your Cloudflare account:
wrangler login
  1. Update wrangler.toml with your account details if needed.

Development

Run locally:

npm run dev

This starts the Worker on http://localhost:8787

Test the frontend: Open public/index.html in your browser, or run:

npx serve public

Deployment

Deploy the Worker:

npm run deploy

Deploy the Pages frontend:

npm run deploy:pages

Update the frontend: After deploying the Worker, update the WORKER_URL in public/index.html with your deployed Worker URL

📝 Usage Examples

Start Monitoring

monitor example.com
monitor https://news.ycombinator.com

Check Status

status example.com

Stop Monitoring

stop example.com

Get Help

help

🧩 Key Code Snippets

1. Durable Object - Alarm Handler (Autonomous Check)

async alarm(): Promise<void> {
  const url = await this.state.storage.get<string>('url');

  // Fetch website
  const response = await fetch(url);
  const status = response.status;

  if (status !== 200) {
    await this.sendAlert({
      type: 'status_error',
      message: `Website ${url} returned status ${status}`
    });
    return;
  }

  // Check for content changes
  const newHtml = await response.text();
  const newHash = await this.hashContent(newHtml);
  const previousHash = await this.state.storage.get<string>('contentHash');

  if (newHash !== previousHash) {
    // Analyze with LLM
    const analysis = await this.analyzeContentChange(previousHtml, newHtml);
    await this.sendAlert({
      type: 'content_change',
      message: `Content changed: ${analysis}`
    });
  }

  // Schedule next check
  await this.state.storage.setAlarm(Date.now() + 10 * 60 * 1000);
}

2. LLM Integration (Llama 3.3)

private async analyzeContentChange(oldHtml: string, newHtml: string): Promise<string> {
  const prompt = `You are a website analyst. Compare the following two versions
of a webpage's HTML. Summarize the semantic difference in one concise sentence.

Old HTML: ${oldHtml.substring(0, 5000)}
New HTML: ${newHtml.substring(0, 5000)}`;

  const response = await this.env.AI.run('@cf/meta/llama-3.3-70b-instruct-fp8-fast', {
    prompt,
    max_tokens: 150,
    temperature: 0.3
  });

  return response.response.trim();
}

3. Agent SDK - Command Routing

const agent = new Agent({
	name: 'WebsiteMonitorAgent',
	model: env.AI,
	systemPrompt: `You are a website monitoring assistant...`
})

// Interpret command
const interpretation = await interpretCommand(message, env)

if (interpretation.action === 'monitor') {
	// Get or create Durable Object for this URL
	const hostname = new URL(url).hostname
	const doId = env.WEBSITE_MONITOR.idFromName(hostname)
	const stub = env.WEBSITE_MONITOR.get(doId)

	// Start monitoring
	await stub.fetch('http://do/start', {
		method: 'POST',
		body: JSON.stringify({ url, realtimeSessionId })
	})
}

4. Frontend - Realtime Communication

// Send command to agent
async function sendMessage() {
	const response = await fetch(`${WORKER_URL}/agent`, {
		method: 'POST',
		headers: { 'Content-Type': 'application/json' },
		body: JSON.stringify({
			message: userInput,
			sessionId: sessionId
		})
	})

	const result = await response.json()
	displayResponse(result)
}

// Receive proactive alerts
function handleIncomingAlert(alert) {
	addMessage(`🔔 ${alert.message}`, 'alert')

	// Browser notification
	new Notification('Website Monitor Alert', {
		body: alert.message
	})
}

🔧 Configuration

Monitoring Interval

Default: 10 minutes. Change in src/durable-objects/WebsiteMonitor.ts:

checkIntervalMinutes: 10 // Change to desired minutes

LLM Model

Currently using: @cf/meta/llama-3.3-70b-instruct-fp8-fast

To change, update in src/durable-objects/WebsiteMonitor.ts:

await this.env.AI.run('@cf/meta/llama-3.3-70b-instruct-fp8-fast', { ... })

🎯 Key Features

  • Autonomous: No user intervention needed after setup
  • Proactive: Alerts pushed to user in real-time
  • Intelligent: AI analyzes semantic changes, not just diffs
  • Scalable: Each website gets isolated DO instance
  • Persistent: State survives Worker restarts
  • Real-time: Bi-directional WebSocket communication

📚 Cloudflare Services Used

  1. Workers - Serverless compute
  2. Workers AI - LLM inference (Llama 3.3)
  3. Agents SDK - Orchestration framework
  4. Durable Objects - Stateful compute with alarms
  5. Pages - Static site hosting
  6. Realtime - WebSocket messaging (conceptual integration)

🐛 Troubleshooting

TypeScript Errors During Development: The TypeScript errors you see are expected in local development. Cloudflare Workers runtime provides these globals. The code will work correctly when deployed.

WebSocket Connection: The current implementation uses HTTP polling for simplicity. For production, integrate the official Cloudflare Realtime SDK once it's available in your account.

Alarm Not Firing: Ensure your Worker has Durable Objects enabled and properly configured in wrangler.toml.

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages