diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/agents/hack-compliance-validator.agent.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/agents/hack-compliance-validator.agent.md new file mode 100644 index 000000000..f060f6f2b --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/agents/hack-compliance-validator.agent.md @@ -0,0 +1,62 @@ +--- +description: "Use when you want to validate that this hack is fully compliant with the microsoft/MicroHack platform conventions. Checks directory naming, deploy-lab.ps1 parameter contract, lab-defaults.json schema validity, platform integration rules (no Connect-AzAccount, no premature RG creation), credential return pattern, and content structure. Run before committing or requesting platform access." +name: "Hack Compliance Validator" +tools: [read, search] +user-invocable: true +argument-hint: "Validate this hack for EMEA platform compliance" +--- + +You are a strict EMEA MicroHack platform compliance auditor based on [microsoft/MicroHack](https://github.com/microsoft/MicroHack). +Inspect this workspace and produce a pass/fail report against every mandatory convention. + +## Compliance Checklist + +Work through each item. Mark ✅ PASS or ❌ FAIL with a brief reason. + +### 1. Directory Structure +- [ ] `labautomation/` directory exists (no dashes — exactly `labautomation`). +- [ ] `labautomation/` contains **only** `deploy-lab.ps1`, `lab-defaults.json`, `README.md`, plus the files `deploy-lab.ps1` itself uses (`shared-deploy-lab.ps1`, Bicep/ARM templates) — no stray files (facilitator notes, runbooks, secrets, etc.). One-time setup / prerequisite docs belong in `walkthrough/` or a separate top-level folder, not here. +- [ ] `labautomation/lab-defaults.json` exists. +- [ ] `labautomation/README.md` exists. +- [ ] A root readme exists **and its casing matches every link that targets it**. GitHub is case-sensitive: if files in `challenges/` or `walkthrough/` link to `../Readme.md`, the root file MUST be `Readme.md` (not `README.md`), and vice-versa. Flag any mismatch as **blocking** — the "Home" navigation links 404 on GitHub even though they resolve on Windows. +- [ ] `challenges/` directory exists. +- [ ] `walkthrough/` directory exists. + +### 2. deploy-lab.ps1 — Parameter Contract +- [ ] `labautomation/deploy-lab.ps1` exists (optional but recommended). +- [ ] If it exists: parameter block declares exactly `$DeploymentType`, `$SubscriptionId`, `$ResourceGroupName`, `$PreferredLocation`, `$AllowedEntraUserIds`. +- [ ] `$DeploymentType` has `[ValidateSet('subscription','resourcegroup','resourcegroup-with-subscriptionowner')]`. + +### 3. deploy-lab.ps1 — Platform Integration Rules +- [ ] Script does **NOT** call `Connect-AzAccount` (platform pre-sets Az context). +- [ ] For `resourcegroup` deployment type: script does **NOT** call `New-AzResourceGroup` (platform pre-creates it). +- [ ] For `subscription` deployment type: script uses `Get-MhhStableHash` to generate a deterministic RG name. +- [ ] Any `Get-MhhStableHash` call passes `-Length` within the valid **12–64** range. Values below 12 throw at runtime and silently fail the whole deployment — flag as **blocking**. + +### 4. deploy-lab.ps1 — Credential Return +- [ ] Script emits at least one `@{ HackboxCredential = @{ name = ...; value = ...; note = ... } }` to the output stream. + +### 5. lab-defaults.json — Schema Validity +- [ ] File is valid JSON (no comments, no trailing commas). +- [ ] Contains `"$schema"` pointing to the MicroHack schema URL. +- [ ] `deploymentType` is one of: `"resourcegroup"`, `"resourcegroup-with-subscriptionowner"`, `"subscription"`. +- [ ] `preferredLocation` is a comma-separated string of Azure region names. +- [ ] `estimatedDailyCostsUsd` is a non-negative number. + +### 6. Platform Compatibility +- [ ] No hard-coded corporate network, specific DNS, or environment-specific blocks. +- [ ] Script works for on-site, online, and hybrid event formats. + +--- + +## How to Run + +1. Use `#tool:search` and `#tool:read` to locate and read each required file. +2. For each checklist item, read the relevant section and determine pass/fail. +3. Output the full checklist with ✅/❌ status. +4. Output a **Summary** section: + - Total: X/6 categories passing + - Blocking issues (❌ on categories 1–5 are blocking) + - Recommendations (non-blocking improvements) + +Be strict. A missing `$schema` in lab-defaults.json or a wrong parameter name in deploy-lab.ps1 causes the platform to skip the script silently — flag these as blocking. A root-readme casing mismatch and an out-of-range `Get-MhhStableHash -Length` are also blocking (broken GitHub links / runtime failure respectively). diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/copilot-instructions.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/copilot-instructions.md new file mode 100644 index 000000000..3f306934a --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/copilot-instructions.md @@ -0,0 +1,80 @@ +# MicroHack Lab — EMEA Platform Conventions + +This workspace is a microhack lab for the **EMEA MicroHack platform** ([microsoft/MicroHack](https://github.com/microsoft/MicroHack)). +The platform renders a front-end UI, but **all resource provisioning comes from `labautomation/deploy-lab.ps1`**. +Without a working deployment script, no resources appear in the Azure portal — the platform is just a shiny front end. + +## Mandatory Structure + +``` +labautomation/ ← exactly this name, no dashes + deploy-lab.ps1 ← platform entry point (optional but recommended) + lab-defaults.json ← platform configuration (required if folder exists) + README.md ← usage notes +challenges/ ← challenge instructions (Markdown) +walkthrough/ ← solutions +README.md ← hack intro, objectives, prerequisites +``` + +## deploy-lab.ps1 — Parameter Contract + +The platform **skips the script** if the parameter block does not exactly match: + +```powershell +param( + [Parameter(Mandatory=$true)] + [ValidateSet('subscription','resourcegroup','resourcegroup-with-subscriptionowner')] + [string]$DeploymentType, + + [Parameter(Mandatory=$true)] + [string]$SubscriptionId, + + [string]$ResourceGroupName = "", + + [string[]]$PreferredLocation = @(), + + [string[]]$AllowedEntraUserIds = @() +) +``` + +## Platform Guarantees (before your script runs) + +- Az context is already set to `$SubscriptionId` — **do NOT call `Connect-AzAccount`** +- For `resourcegroup`/`resourcegroup-with-subscriptionowner`: the RG exists and your script has `Owner` — **do NOT call `New-AzResourceGroup`** +- For `subscription`: your script has `Owner` on the subscription; **you must create the RG yourself** using `Get-MhhStableHash` for deterministic naming +- `Az.Accounts` and `Az.Resources` modules are already imported + +## lab-defaults.json — Required Shape + +```json +{ + "$schema": "https://raw.githubusercontent.com/microsoft/MicroHack/refs/heads/main/lab-defaults-schema.json", + "groups": [], + "deploymentType": "resourcegroup", + "labsPerSubscription": 4, + "preferredLocation": "westeurope, swedencentral, norwayeast", + "estimatedDailyCostsUsd": 5.0 +} +``` + +- `deploymentType`: `"resourcegroup"` | `"resourcegroup-with-subscriptionowner"` | `"subscription"` +- `groups`: `[]` (Azure only), `["GHCPUsers"]` (GitHub Copilot seat), `["M365-E5-Users"]` (M365 E5) +- `preferredLocation`: comma-separated regions, priority order +- `estimatedDailyCostsUsd`: per-user per-day cost for the lifecycle wizard + +## Returning Credentials to Users + +Write a hashtable to the output stream — the platform captures it and shows it on the user's dashboard: + +```powershell +@{ HackboxCredential = @{ name = "AdminPassword"; value = "TopSecret!"; note = "VM admin password" } } +``` + +## Delivery Deadline + +Labs must have at least a skeleton committed by **end of July** — September events are already scheduled. + +## Platform Compatibility + +The platform supports **on-site**, **online**, and **hybrid** delivery. +Do not hard-code corporate network or specific DNS assumptions. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/instructions/labautomation.instructions.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/instructions/labautomation.instructions.md new file mode 100644 index 000000000..699ded2dd --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/instructions/labautomation.instructions.md @@ -0,0 +1,101 @@ +--- +description: "Use when creating, editing, or reviewing labautomation/deploy-lab.ps1, lab-defaults.json, or any PowerShell helper inside labautomation/. Enforces microsoft/MicroHack platform deployment conventions." +applyTo: "labautomation/**" +--- + +# Lab Automation Conventions (microsoft/MicroHack) + +## deploy-lab.ps1 — Required Parameter Contract + +The platform **silently skips** the script if the parameter block does not match exactly: + +```powershell +param( + [Parameter(Mandatory=$true)] + [ValidateSet('subscription','resourcegroup','resourcegroup-with-subscriptionowner')] + [string]$DeploymentType, + + [Parameter(Mandatory=$true)] + [string]$SubscriptionId, + + [string]$ResourceGroupName = "", + + [string[]]$PreferredLocation = @(), + + [string[]]$AllowedEntraUserIds = @() +) +``` + +## Critical Platform Rules + +**NEVER do these — they break platform integration:** + +```powershell +# WRONG — platform already sets Az context +Connect-AzAccount + +# WRONG — platform pre-creates the RG for resourcegroup deployments +New-AzResourceGroup -Name $ResourceGroupName ... +``` + +**For `subscription` deployments only** — you must create your own RG using a deterministic name: + +```powershell +$stableHash = Get-MhhStableHash $AllowedEntraUserIds -Length 24 +$effectiveResourceGroup = "lab-$stableHash" +New-AzResourceGroup -Name $effectiveResourceGroup -Location $effectiveLocation +``` + +## Resolving Effective Location + +```powershell +$effectiveLocation = if ($PreferredLocation.Count -gt 0) { $PreferredLocation[0] } else { "westeurope" } +``` + +## Deploying Resources + +Reference templates relative to `$scriptPath`: + +```powershell +$scriptPath = Split-Path -Parent $MyInvocation.MyCommand.Definition +$template = Join-Path $scriptPath "template.bicep" +New-AzResourceGroupDeployment -ResourceGroupName $effectiveResourceGroup -TemplateFile $template -Verbose +``` + +## Returning Credentials to the User Dashboard + +Write a hashtable to the output stream — the platform captures every one: + +```powershell +@{ HackboxCredential = @{ name = "AdminPassword"; value = "TopSecret!"; note = "VM admin password" } } +``` + +Always return at minimum the resource group name so users can find their resources. + +## lab-defaults.json — Required Shape + +```json +{ + "$schema": "https://raw.githubusercontent.com/microsoft/MicroHack/refs/heads/main/lab-defaults-schema.json", + "groups": [], + "deploymentType": "resourcegroup", + "labsPerSubscription": 4, + "preferredLocation": "westeurope, swedencentral, norwayeast", + "estimatedDailyCostsUsd": 5.0 +} +``` + +- `$schema` is **required** — the platform validates against it +- `deploymentType`: `"resourcegroup"` | `"resourcegroup-with-subscriptionowner"` | `"subscription"` +- `groups`: `[]` for Azure-only; add `"GHCPUsers"` or `"M365-E5-Users"` as needed +- `preferredLocation`: comma-separated regions in priority order — list multiple for regional fallback +- `estimatedDailyCostsUsd`: cost per user per day; used in the lifecycle cost wizard + +## Available Platform Helper Cmdlets + +| Cmdlet | Use | +|--------|-----| +| `Get-MhhStableHash` | Deterministic per-user hash for resource naming in `subscription` mode | +| `Get-MhhLabUser` | Get Entra user details for `$AllowedEntraUserIds` | +| `Invoke-MhhDeploymentWithRegionFallback` | Deploy with automatic region fallback | +| `Test-MhhDeploymentFailureRetryable` | Check if a deployment error is transient | diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/prompts/scaffold-lab-automation.prompt.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/prompts/scaffold-lab-automation.prompt.md new file mode 100644 index 000000000..6582a2ef9 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.github/prompts/scaffold-lab-automation.prompt.md @@ -0,0 +1,86 @@ +--- +description: "Scaffold a complete labautomation/ directory for a new EMEA MicroHack. Generates deploy-lab.ps1, lab-defaults.json, and README.md that are 100% compliant with the microsoft/MicroHack platform contract." +name: "Scaffold Lab Automation" +agent: "agent" +argument-hint: "Describe the Azure resources this hack needs to deploy (e.g. 'a Storage Account and a Function App for an inventory management scenario')" +tools: [edit, read, search] +--- + +You are scaffolding the `labautomation/` directory for an EMEA MicroHack. +The user has described what Azure resources the hack needs. + +**Do not ask clarifying questions** — infer reasonable defaults and note assumptions in the README. + +## Step 1 — Read existing context + +Check whether `labautomation/` already exists. If it does, read existing files before overwriting. + +## Step 2 — Create lab-defaults.json + +Create `labautomation/lab-defaults.json`. Always include the `$schema` field: + +```json +{ + "$schema": "https://raw.githubusercontent.com/microsoft/MicroHack/refs/heads/main/lab-defaults-schema.json", + "groups": [], + "deploymentType": "resourcegroup", + "labsPerSubscription": 4, + "preferredLocation": "westeurope, swedencentral, norwayeast", + "estimatedDailyCostsUsd": 5.0 +} +``` + +Adjust `deploymentType`, `preferredLocation`, and `estimatedDailyCostsUsd` to match the hack's needs. +Add `"GHCPUsers"` or `"M365-E5-Users"` to `groups` only if the hack requires those licenses. + +## Step 3 — Create deploy-lab.ps1 + +Create `labautomation/deploy-lab.ps1` following the **exact** platform parameter contract: + +```powershell +param( + [Parameter(Mandatory=$true)] + [ValidateSet('subscription','resourcegroup','resourcegroup-with-subscriptionowner')] + [string]$DeploymentType, + + [Parameter(Mandatory=$true)] + [string]$SubscriptionId, + + [string]$ResourceGroupName = "", + + [string[]]$PreferredLocation = @(), + + [string[]]$AllowedEntraUserIds = @() +) +``` + +**Critical rules — the platform silently skips the script if these are violated:** +- Do NOT call `Connect-AzAccount` — Az context is pre-set by the platform +- Do NOT call `New-AzResourceGroup` for `resourcegroup` deployments — the RG is pre-created +- For `subscription` deployments: use `Get-MhhStableHash $AllowedEntraUserIds -Length 24` for deterministic RG naming, then call `New-AzResourceGroup` +- Always emit at least one `@{ HackboxCredential = @{ name = ...; value = ...; note = ... } }` to the output stream + +Add the actual resource deployments (Bicep, ARM, or Az cmdlets) in the body. +Reference the Bicep/ARM template file relative to `$scriptPath = Split-Path -Parent $MyInvocation.MyCommand.Definition`. + +## Step 4 — Create README.md + +Create `labautomation/README.md` covering: +- What the script deploys (bullet list of resources) +- `deploymentType` choice and reason +- `preferredLocation` choice and reason +- `estimatedDailyCostsUsd` basis +- Any assumptions made during scaffolding + +## Step 5 — Self-validate + +After creating all files, verify against the compliance checklist: +- Directory named exactly `labautomation` (no dashes) ✅ +- `lab-defaults.json` has `$schema` field ✅ +- `deploy-lab.ps1` parameter block matches contract exactly ✅ +- No `Connect-AzAccount` in the script ✅ +- No `New-AzResourceGroup` for resourcegroup deployment type ✅ +- At least one `HackboxCredential` returned ✅ +- No hard-coded environment assumptions ✅ + +Report any item you could not satisfy and why. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.gitignore b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.gitignore new file mode 100644 index 000000000..41d9ae696 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/.gitignore @@ -0,0 +1,8 @@ +__pycache__/ +*.pyc +.venv/ +tickets.jsonl +sop_index.json +benchmark_results.json +artifacts/ +work/ diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/Readme.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/Readme.md new file mode 100644 index 000000000..65d6c1c97 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/Readme.md @@ -0,0 +1,109 @@ +# **Offline Industrial Copilot - Foundry Local MicroHack** + +- [**MicroHack introduction**](#microhack-introduction) +- [**MicroHack context**](#microhack-context) +- [**Objectives**](#objectives) +- [**MicroHack challenges**](#microhack-challenges) +- [**Contributors**](#contributors) + +# MicroHack introduction + +A capable AI agent can run **entirely offline on the factory floor**: local model, local knowledge and local tools. You build it with familiar cloud tooling and then pull the network cable. + +In this MicroHack you build a **maintenance technician copilot** for a fictional manufacturer, Contoso Precision Machining. A technician on the shop floor asks: + +> *"Spindle vibration on CNC-07 is above the limit. What do I do?"* + +The copilot answers from the plant's standard operating procedures (SOPs) and **cites its source**, looks up the machine status, opens a maintenance ticket, and predicts the likely fault with a model that was trained in the cloud. At the end you disconnect the network and prove that every one of these still works. + +```mermaid +flowchart LR + T([Technician]) --> A[Agent
small language model] + A -->|function calling| S[Machine status
local JSON] + A -->|function calling| K[Maintenance tickets
local file] + A -->|function calling| R[SOP search
local embeddings] + A -->|function calling| F[Fault classifier
ONNX model] + subgraph Device[Laptop or VM - no internet needed] + A + S + K + R + F + FL[Foundry Local
OpenAI-compatible endpoint] --- A + end + C[(Azure Machine Learning
training in the cloud)] -. trained once, then exported .-> F +``` + +This MicroHack is about the **agent**. It is not a deep dive into Kubernetes, Azure Arc or industrial protocols, and you do not need any of them. + +Please read these short articles first (optional, 15 minutes): + +- [What is Foundry Local?](https://learn.microsoft.com/azure/foundry-local/what-is-foundry-local) +- [Use the Foundry Local CLI](https://learn.microsoft.com/azure/foundry-local/how-to/how-to-use-foundry-local-cli) + +# MicroHack context + +The audience is **application and AI developers** who want to build agents that run close to production equipment, where the connection can be slow, expensive or missing. You need basic Python. You do not need to know Kubernetes, Azure Arc or any industrial protocol. + +The day has six hours of challenges. Each challenge builds on the previous one. If a team falls behind, a coach hands out the solution so the team can continue. + +| Time | Challenge | Duration | +|------|-----------|----------| +| 09:00 | Kickoff and scenario | 20 min | +| 09:20 | 1 - Get set up | 20 min | +| 09:40 | 2 - Choose your edge model | 45 min | +| 10:25 | 3 - Build your first local agent | 60 min | +| 11:25 | Lunch | 30 min | +| 11:55 | 4 - Agentic retrieval over local knowledge | 75 min | +| 13:10 | 5 - Train in the cloud, deploy at the edge | 60 min (shortcut 25 min) | +| 14:10 | 6 - Cut the cord | 30 min | +| 14:40 | Wrap-up and discussion | 20 min | + +# Objectives + +After completing this MicroHack you will: + +- Run a small language model and an embedding model on your own device with Foundry Local, and call them with the OpenAI SDK. +- Choose a model with evidence: latency, size, tool calling and answer quality. +- Build an agent that calls tools with function calling, with a person approving every action that changes something. +- Ground an agent in local documents with agentic retrieval, citations and an evaluation set. +- Train a model in Azure Machine Learning, export it to ONNX and run it at the edge as an agent tool. +- Prove that the whole copilot works without an internet connection. + +# MicroHack challenges + +## General prerequisites + +Complete these tasks **before** the session so the time goes to the agent and not to installation. + +- A **laptop or a VM** that runs Foundry Local: + - Windows 11 (x64 or ARM64) or macOS on Apple silicon, with at least 16 GB of RAM and 15 GB of free disk space. + - Your coach can give you a Windows 11 VM instead. It runs the models on its CPU, so answers are slower. + - Foundry Local does not have a command-line tool for Linux yet, so this MicroHack does not use Linux. +- **Python 3.11 or newer**. +- This repository (or a copy of this folder) on your device. +- An **Azure subscription** with permission to create an Azure Machine Learning workspace. Needed for challenge 5 only. If you take the shortcut in challenge 5 you do not need it. + +You can start the model downloads early. They are a few gigabytes in total. + +## Challenges + +* [Challenge 1 - Get set up](challenges/challenge-01.md) **<- Start here** +* [Challenge 2 - Choose your edge model](challenges/challenge-02.md) +* [Challenge 3 - Build your first local agent](challenges/challenge-03.md) +* [Challenge 4 - Agentic retrieval over local knowledge](challenges/challenge-04.md) +* [Challenge 5 - Train in the cloud, deploy at the edge](challenges/challenge-05.md) +* [Challenge 6 - Cut the cord](challenges/challenge-06.md) + +## Solutions - Spoilerwarning + +* [Solution 1 - Get set up](./walkthrough/challenge-01/solution-01.md) +* [Solution 2 - Choose your edge model](./walkthrough/challenge-02/solution-02.md) +* [Solution 3 - Build your first local agent](./walkthrough/challenge-03/solution-03.md) +* [Solution 4 - Agentic retrieval over local knowledge](./walkthrough/challenge-04/solution-04.md) +* [Solution 5 - Train in the cloud, deploy at the edge](./walkthrough/challenge-05/solution-05.md) +* [Solution 6 - Cut the cord](./walkthrough/challenge-06/solution-06.md) + +## Contributors + +* Digvijay Chauhan [GitHub](https://github.com/Digvijay) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-01.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-01.md new file mode 100644 index 000000000..7b7fe9192 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-01.md @@ -0,0 +1,30 @@ +# Challenge 1 - Get set up + +**[Home](../Readme.md)** - [Next Challenge](challenge-02.md) + +## Goal + +Get a small language model running **on your own device** and call it from Python, using the same OpenAI library you would use for a cloud model. + +## Actions + +* Install Foundry Local on your laptop or VM: + * Windows: `winget install Microsoft.FoundryLocal` + * macOS: `brew tap microsoft/foundrylocal` and then `brew install foundrylocal` +* Open a new terminal and check that the `foundry` command works. Find out how to list the models in the catalog. +* Download the model with the alias `phi-4-mini` and chat with it in the terminal. +* Create a folder named `work` inside this lab folder, next to `resources`. Create a Python virtual environment there and install the packages from `resources/starter/requirements.txt`. Copy `resources/starter/fl.py` into `work`. +* Write a short Python script that sends one question to the local model and prints the answer. The helper in `fl.py` gives you the model ID and a ready-to-use OpenAI client. +* Find the address of the local service. What is special about it? + +## Success criteria + +* `foundry` is installed and the model `phi-4-mini` is downloaded. +* Your Python script prints an answer from the local model. +* You can tell, in one sentence, why the address of the service is not reachable from another computer. + +## Learning resources + +* [What is Foundry Local?](https://learn.microsoft.com/azure/foundry-local/what-is-foundry-local) +* [Use the Foundry Local CLI](https://learn.microsoft.com/azure/foundry-local/how-to/how-to-use-foundry-local-cli) +* [Foundry Local CLI reference](https://learn.microsoft.com/azure/foundry-local/reference/reference-cli) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-02.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-02.md new file mode 100644 index 000000000..813a82f49 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-02.md @@ -0,0 +1,33 @@ +# Challenge 2 - Choose your edge model + +[Previous Challenge](challenge-01.md) - **[Home](../Readme.md)** - [Next Challenge](challenge-03.md) + +## Goal + +Choose the small language model for your copilot **with evidence**, not by feeling. What fits on a shop-floor PC matters more than the best benchmark score. + +## Actions + +* Pick two small models from the Foundry Local catalog. Use `phi-4-mini` as one of them. Look at the size, the hardware each model runs on, and whether it supports **tool calling**. +* Write a script that runs both models against the same questions: + * The questions in `resources/data/eval/retrieval_eval.json`. Give the model the matching SOP from `resources/data/sops` and ask it to answer from it. + * The tool-calling cases in `resources/data/eval/tool_eval.json`. +* Record for each model: + * how long the first token takes and how many tokens per second you get, + * how big the model is, and how much memory it uses (Task Manager or Activity Monitor can help), + * how many answers contain the expected facts, + * how many tool-calling cases return the right tool with the right arguments. +* Try tool calling in two ways: `tool_choice="auto"` and `tool_choice="required"`. What changes? +* Pick one model and explain your choice in one or two sentences. + +## Success criteria + +* Both models ran against the full question set and the tool-calling cases. +* You have latency, size, memory, answer-quality and tool-calling figures for each model. +* You can explain your choice in one or two sentences. +* You know which of your two models you must **not** use for the agent, and why. + +## Learning resources + +* [Foundry Local CLI reference](https://learn.microsoft.com/azure/foundry-local/reference/reference-cli) +* [Function calling with Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-03.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-03.md new file mode 100644 index 000000000..3347a7a10 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-03.md @@ -0,0 +1,34 @@ +# Challenge 3 - Build your first local agent + +[Previous Challenge](challenge-02.md) - **[Home](../Readme.md)** - [Next Challenge](challenge-04.md) + +## Goal + +Turn the local model into an agent that can **act**, not only answer. + +## Actions + +* Write a system prompt for the maintenance technician copilot. +* Create two tools in Python: + * `get_machine_status(machine_id)` reads the machine from `resources/data/machines.json`. + * `create_maintenance_ticket(machine_id, summary, priority)` appends a ticket to a local file. The priority is `low`, `medium` or `high`. +* Describe both tools to the model with JSON schemas and call the local endpoint with `tools=...`. +* Write the **tool-call loop**: send the request, run the tools the model asks for, send the results back, and repeat until the model gives a final answer. +* Test it with the cases in `resources/data/eval/tool_eval.json`, for example: + * *"What is the status of machine CNC-07?"* + * *"Open a high priority ticket for CNC-07: spindle vibration above the stop limit."* + * *"Why is the sky blue?"* (the agent must not create a ticket for this) +* Look at what the model sends as tool arguments. Can it send something wrong? What does your code do about that? +* Add a **confirmation step**: before the agent creates a ticket, the technician must say yes. + +## Success criteria + +* The agent calls the machine status tool when you ask about a specific machine. +* The agent creates a local ticket when you ask for one, and **only** after the technician confirms. +* The tool-call loop ends with a final answer. +* Asking about an unknown machine gives a clear answer and not a crash. + +## Learning resources + +* [Function calling with Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling) +* [Foundry Local CLI reference](https://learn.microsoft.com/azure/foundry-local/reference/reference-cli) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-04.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-04.md new file mode 100644 index 000000000..7553b39dc --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-04.md @@ -0,0 +1,30 @@ +# Challenge 4 - Agentic retrieval over local knowledge + +[Previous Challenge](challenge-03.md) - **[Home](../Readme.md)** - [Next Challenge](challenge-05.md) + +## Goal + +Ground the agent in the plant's SOPs, with the **agent deciding when and what to retrieve**, and cite the source of every answer. + +## Actions + +* Download the embedding model `qwen3-embedding-0.6b` and load it next to your chat model. Do both fit on your device at the same time? +* Cut the SOPs in `resources/data/sops` into chunks. Think about what a good chunk is: where do you cut, and what must travel with every chunk so the answer can cite it? +* Create an embedding for every chunk and keep them in a simple local store (a file and NumPy are enough). +* Give the agent a **`search_sops` tool**. The agent decides when to call it. +* Make every grounded answer show **which SOP** it came from. Can you trust the model to do this on its own? +* Make sure the agent says *"I could not find this in the SOPs"* when the answer is not there, for example for *"What is on the cafeteria menu on Friday?"* +* Make a question with **two topics** work, for example: *"What torque do I use for the spindle bearing bolts, and what must I do before I start work on the spindle?"* +* Evaluate your agent with `resources/data/eval/retrieval_eval.json` and `compound_eval.json`. Report: correct facts, correct source, and correct refusals. Then change one thing and see if a number improves. + +## Success criteria + +* *"At what vibration level must a CNC spindle be stopped?"* is answered from SOP-CNC-001 **with a citation**. +* The agent retrieves for each topic when a question has more than one topic. +* A question that is not in the SOPs gets a refusal and not an invented answer. +* You can report your results on the evaluation questions, and you changed one thing and measured the effect. + +## Learning resources + +* [What is Foundry Local?](https://learn.microsoft.com/azure/foundry-local/what-is-foundry-local) +* [Foundry Local CLI reference](https://learn.microsoft.com/azure/foundry-local/reference/reference-cli) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-05.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-05.md new file mode 100644 index 000000000..936f1ff90 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-05.md @@ -0,0 +1,43 @@ +# Challenge 5 - Train in the cloud, deploy at the edge + +[Previous Challenge](challenge-04.md) - **[Home](../Readme.md)** - [Next Challenge](challenge-06.md) + +## Goal + +Show that the edge agent can use a model that you **trained with cloud tooling** and that then runs locally, with no cloud dependency. + +Your coach tells you which path to take. The result is the same. + +## Actions + +### Full path (60 minutes) + +* In your Azure Machine Learning workspace (the name is on your lab page), create a small compute instance and open a notebook. +* Upload `resources/data/sensor_history.csv` and train a **fault classifier**: from the sensor readings it predicts `normal`, `bearing_wear`, `overheating` or `low_lubrication`. +* Export the model to **ONNX** and check that ONNX Runtime gives the same answers as your training library. +* Download the ONNX file to your device. +* Stop or delete the compute instance when you are done. + +### Shortcut path (25 minutes) + +* Read through the notebook `resources/shortcut/train_fault_classifier.ipynb` to see how the model was trained and exported. You do not need to run it. +* Use the ready-made `resources/shortcut/fault_classifier.onnx`. + +### Both paths + +* Run the ONNX model on your device with ONNX Runtime, **without** the training library. How long does one prediction take? +* Add a `predict_fault(machine_id)` tool to your agent. It reads the machine's readings and returns the predicted fault with its confidence. +* Ask the agent for the most likely fault of CNC-07 in **three different ways**. Does it call your classifier every time? Then make the choice of tool reliable. Measure it with the requests in `resources/data/eval/routing_eval.json`, and then add your own. +* Look at CNC-07, CMP-01 and PRESS-02. Which prediction would you trust, and which not? + +## Success criteria + +* The ONNX model runs on your device and does not call the cloud. +* The agent calls the model as a tool and uses the result in its answer. +* You can say how confident the classifier is for each of the three machines, and what a technician should do with a low-confidence result. + +## Learning resources + +* [Azure Machine Learning notebooks](https://learn.microsoft.com/azure/machine-learning/how-to-run-jupyter-notebooks) +* [ONNX Runtime](https://onnxruntime.ai/docs/) +* [Convert scikit-learn models to ONNX](https://onnx.ai/sklearn-onnx/) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-06.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-06.md new file mode 100644 index 000000000..fb3312284 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/challenge-06.md @@ -0,0 +1,33 @@ +# Challenge 6 - Cut the cord + +[Previous Challenge](challenge-05.md) - **[Home](../Readme.md)** - [Finish](finish.md) + +## Goal + +**Prove** that the complete copilot works with no internet connection. + +## Actions + +* Run your complete copilot once **while connected** and write down how long each kind of question takes: a procedure question, a machine status, a fault prediction, and a ticket. +* **Get ready before you disconnect.** Make sure the Foundry Local service is running and your models are loaded, and ask one question of each kind so everything is warm. **Do not restart the service while you are offline.** Find out why: what does the service need when it starts? +* Disconnect the device from the internet. Choose a way that matches your setup: + * Laptop: turn off Wi-Fi, unplug the cable, or block outbound traffic with the firewall. + * VM: block outbound internet traffic with a network security group rule. Keep a way to reach the VM. +* Check that you really are offline. +* Run the same scenario again: answer from an SOP with a citation, call the tools, use the ONNX model, and create a ticket. +* Record the latency of each step while offline. +* Find out whether **anything** still needs the internet. Look at what your copilot does at start-up, and what the Foundry Local service does. +* Discuss what must still happen when the device is back online: updates, new models, syncing tickets, monitoring. +* **Stretch:** make your copilot survive a restart of the model service while the device is offline. The Foundry Local SDK (`pip install foundry-local-sdk`) can start models from your local cache. What does it need that the `foundry` service gets from the internet? + +## Success criteria + +* Every capability from challenges 3 to 5 works while the device is disconnected. +* You recorded the latency of each step, connected and disconnected, and can compare them. +* You can name at least two things that still need the cloud, and say when they would happen. + +## Learning resources + +* [Foundry Local architecture](https://learn.microsoft.com/azure/foundry-local/concepts/foundry-local-architecture) +* [Network security groups](https://learn.microsoft.com/azure/virtual-network/network-security-groups-overview) +* [Windows Firewall rules with PowerShell](https://learn.microsoft.com/powershell/module/netsecurity/new-netfirewallrule) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/finish.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/finish.md new file mode 100644 index 000000000..cce0900fd --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/challenges/finish.md @@ -0,0 +1,21 @@ +# Finish + +[Previous Challenge](challenge-06.md) - **[Home](../Readme.md)** + +## Congratulations + +You finished the MicroHack *Offline Industrial Copilot*. You built, measured and disconnected a working agent yourself: a local model, local knowledge, local tools and a model trained in the cloud, running on the factory floor without a network. + +Take these questions back to your own project: + +* Which of your agent's tasks must work without a connection, and which can wait for one? +* How small can the model be before the tool calls stop being reliable? +* Where does your agent need a person to say yes? + +## Where to go next + +In this MicroHack you built every layer yourself on one device. If your customer runs Azure Local and wants a managed version of the same ideas (document ingestion, vector search, agents, an MCP server and a chat interface), read [Agentic Retrieval and Agents and Tools with Foundry Local](https://learn.microsoft.com/azure/azure-arc/agents-tools-foundry-local/overview). It runs on Azure Arc-enabled Kubernetes and is in preview. It is a cluster platform and not the on-device runtime used here, so check how the two relate before you compare them. + +If you want to give feedback, please open an issue on the repository or get in touch with one of us directly. + +Thank you for investing the time and see you next time! diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/README.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/README.md new file mode 100644 index 000000000..db937cd09 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/README.md @@ -0,0 +1,45 @@ +# Lab automation + +Deploys the per-participant resources for the **Offline Industrial Copilot** MicroHack. + +## What it deploys + +| Resource | Why | +|----------|-----| +| Azure Machine Learning workspace (with storage account, key vault, Log Analytics and Application Insights) | Challenge 5, full path: train the fault classifier in the cloud | +| Windows 11 Pro VM with Python 3.12, a public IP and RDP (optional) | For participants who do not bring a laptop that can run Foundry Local | + +Foundry Local itself is **not** installed by the automation. Installing it is part of challenge 1. + +## Files + +| File | Purpose | +|------|---------| +| `deploy-lab.ps1` | Platform entry point, one run per participant | +| `shared-deploy-lab.ps1` | Registers resource providers and the public IP feature once per subscription | +| `main.bicep` | The resources above | +| `lab-defaults.json` | Platform configuration | + +## Decisions + +- **`deploymentType: resourcegroup`**: the platform creates the resource group and the participant is its owner. Nothing in the lab needs subscription-level rights. +- **`preferredLocation: swedencentral, westeurope, norwayeast`**: regions with Azure Machine Learning and the `Standard_D8s_v5` VM size. The platform falls back to the next region if a deployment fails. +- **`labsPerSubscription: 4`**: each lab uses one 8-vCPU VM, so four labs use 32 vCPUs of the DSv5 family. +- **`estimatedDailyCostsUsd: 8`**: about 6 USD for the VM (8 hours), plus the disk, the public IP and a small Azure Machine Learning compute for challenge 5. The workspace itself costs nothing while idle. +- **VM is Windows, not Linux**: the Foundry Local command-line tool is supported on Windows and macOS. The VM runs the models on its CPU, so answers are slower than on a laptop with a GPU or NPU. Choose a GPU `vmSize` in `main.bicep` for faster answers. +- **The extension installs the Visual C++ runtime** next to Python. Foundry Local cannot start without it on a fresh Windows image. +- **Set `$deployVm = $false`** in `deploy-lab.ps1` if every participant brings a suitable laptop. +- **`shared-deploy-lab.ps1` registers `Microsoft.Network/AllowBringYourOwnPublicIpAddress`**, like other labs on the platform that give a VM a public IP. Without it some subscriptions refuse every public IP with `SubscriptionNotRegisteredForFeature`, and the platform stops the lab without trying another region. The `SecurityControl: Ignore` tag does not help here: this is a subscription feature, not a policy. The script also registers `Microsoft.Network` again, because the feature only takes effect after that. +- **The VM password comes from `New-MhhStablePassword -Purpose 'vm-admin'`**: the same value on every run. A redeploy keeps the first password on an existing VM, so a new random password would put a password on the dashboard that does not work. + +## Credentials returned to the participant + +Resource group, Azure Machine Learning workspace, region, and (with a VM) the VM address, user name and password. The password stays the same on every run of the script. + +## Assumptions and checks + +- `Invoke-MhhDeploymentWithRegionFallback`, `Get-MhhStableHash` and `New-MhhStablePassword` are platform helpers. Both scripts were tested **unchanged** with the helper module from the platform's local-testing image (`ghcr.io/qxsch/adminpwsh`): `shared-deploy-lab.ps1` registered the public IP feature on a subscription that refused public IPs (1.4 minutes), then `deploy-lab.ps1` provisioned the lab in `swedencentral` on the first attempt in 6.3 minutes, with the VM address, user and password on the dashboard. After provisioning the VM had Python 3.12.8 and the Visual C++ runtime. +- On that VM, signed in as `labadmin`, Foundry Local 0.12.1, the models and the lab code for challenges 1 to 6 were run, including the offline run with the internet blocked (see the coach notes in the walkthrough), and so was the Azure Machine Learning workspace (a compute instance and the challenge 5 notebook, accuracy 0.94). +- **Azure Machine Learning workspaces are soft-deleted** when their resource group is deleted, and the workspace name is the same for a participant on every run. If the platform wipes and recreates the resource group (a retry or a region fallback), the new deployment can meet the soft-deleted name. The platform purges it and treats this as a same-region retry; the older helper in the testing image treats it as fatal. When you test locally with that image, use a new participant ID or wait for the purge. +- Not verified: a participant signing in over Remote Desktop, and the platform's current helper (only the version in the testing image). +- The VM is open for Remote Desktop from the internet and carries the `SecurityControl: Ignore` tag, like other labs on the platform. Do not use it for anything except this lab. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/deploy-lab.ps1 b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/deploy-lab.ps1 new file mode 100644 index 000000000..b7fe0703e --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/deploy-lab.ps1 @@ -0,0 +1,96 @@ +<# +.SYNOPSIS +Deploys the per-participant resources for the Offline Industrial Copilot MicroHack. +.DESCRIPTION +Deploys an Azure Machine Learning workspace (challenge 5: train in the cloud) and an optional Windows 11 VM +that participants can use instead of their own laptop. Foundry Local itself is installed by the participant in challenge 1. +.PARAMETER DeploymentType +Defines the deployment scope; allowed values are subscription or resourcegroup. +.PARAMETER SubscriptionId +Specifies the Azure subscription that contains the lab resources. +.PARAMETER ResourceGroupName +In case of resourcegroup deployment, specifies the target resource group name. +.PARAMETER PreferredLocation +Specifies the preferred Azure regions (ordered by preference) for resource deployment. An empty array indicates no preference. +.PARAMETER AllowedEntraUserIds +Entra user object IDs of the participant who owns this lab. +#> +param( + [Parameter(Mandatory=$true)] + [ValidateSet('subscription','resourcegroup', 'resourcegroup-with-subscriptionowner')] + [string]$DeploymentType, + + [Parameter(Mandatory=$true)] + [string]$SubscriptionId, + + [string]$ResourceGroupName = "", + + [string[]]$PreferredLocation = @(), + + [string[]]$AllowedEntraUserIds = @() +) + +$scriptPath = Split-Path -Parent $MyInvocation.MyCommand.Definition + +# Set to $false if every participant brings a laptop that can run Foundry Local. +$deployVm = $true + +if($DeploymentType -eq 'resourcegroup' -and [string]::IsNullOrEmpty($ResourceGroupName)) { + throw "ResourceGroupName must be provided when DeploymentType is 'resourcegroup'." +} + +if($PreferredLocation.Count -gt 0) { + $effectiveLocation = $PreferredLocation[0] +} else { + $effectiveLocation = "swedencentral" +} + +if($DeploymentType -eq 'subscription') { + $stableHash = Get-MhhStablehash $AllowedEntraUserIds -Length 24 + $effectiveResourceGroup = "lab-$stableHash" + Write-Host "Deploying lab resources at the subscription level in subscription $SubscriptionId..." + if(-not (Get-AzResourceGroup -Name $effectiveResourceGroup -ErrorAction SilentlyContinue)) { + Write-Host "Creating resource group '$effectiveResourceGroup' in location '$effectiveLocation'..." + New-AzResourceGroup -Name $effectiveResourceGroup -Location $effectiveLocation -Verbose + } else { + Write-Host "Resource group '$effectiveResourceGroup' already exists." + } +} else { + $effectiveResourceGroup = $ResourceGroupName +} + +if($AllowedEntraUserIds.Count -eq 0) { + throw "At least one AllowedEntraUserIds value is required." +} + +$resourceSuffix = (Get-MhhStablehash $AllowedEntraUserIds -Length 12).ToLower() + +# Same password on every run for this lab, so the one on the dashboard keeps working after a re-run or a region fallback. +$vmPassword = New-MhhStablePassword -Purpose 'vm-admin' + +@{"HackboxCredential" = @{ name = "ResourceGroupName"; value = $effectiveResourceGroup; note = "The resource group of your lab" }} +@{"HackboxCredential" = @{ name = "MachineLearningWorkspace"; value = "mlw-oic-$resourceSuffix"; note = "Azure Machine Learning workspace for challenge 5" }} + +$template = Join-Path $scriptPath "main.bicep" +$templateParameters = @{ + resourceSuffix = $resourceSuffix + deployVm = $deployVm + vmAdminPassword = $vmPassword +} + +$deployment = Invoke-MhhDeploymentWithRegionFallback ` + -PreferredLocations $PreferredLocation ` + -ResourceGroupName $effectiveResourceGroup ` + -RgOwnerEntraObjectIds $AllowedEntraUserIds ` + -Tag @{ CostControl = "Ignore"; SecurityControl = "Ignore" } ` + -TemplateFile $template ` + -TemplateParameterObject $templateParameters + +@{"HackboxCredential" = @{ name = "Region"; value = $deployment.LocationUsed; note = "The Azure region the lab deployment succeeded in" }} + +if($deployVm) { + $outputs = $deployment.Outputs + @{"HackboxCredential" = @{ name = "VmAddress"; value = $outputs["vmPublicIp"]; note = "Public IP of your Windows 11 VM (connect with Remote Desktop)" }} + @{"HackboxCredential" = @{ name = "VmUser"; value = $outputs["vmAdminUser"]; note = "VM user name" }} + @{"HackboxCredential" = @{ name = "VmPassword"; value = $vmPassword; note = "VM password" }} +} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/lab-defaults.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/lab-defaults.json new file mode 100644 index 000000000..f759094b4 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/lab-defaults.json @@ -0,0 +1,8 @@ +{ + "$schema": "https://raw.githubusercontent.com/microsoft/MicroHack/refs/heads/main/lab-defaults-schema.json", + "groups": [], + "deploymentType": "resourcegroup", + "labsPerSubscription": 4, + "preferredLocation": "swedencentral, westeurope, norwayeast", + "estimatedDailyCostsUsd": 8.0 +} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/main.bicep b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/main.bicep new file mode 100644 index 000000000..c4af4d99c --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/main.bicep @@ -0,0 +1,211 @@ +targetScope = 'resourceGroup' + +@description('Azure region for all resources.') +param location string = resourceGroup().location + +@description('Short unique suffix used in resource names.') +@minLength(4) +@maxLength(12) +param resourceSuffix string + +@description('Deploy a Windows 11 VM that participants can use instead of their own laptop.') +param deployVm bool = true + +@description('VM size. Foundry Local runs on the CPU of this size. Pick a GPU size for faster answers.') +param vmSize string = 'Standard_D8s_v5' + +@description('VM administrator user name.') +param vmAdminUser string = 'labadmin' + +@secure() +@description('VM administrator password. Generated by deploy-lab.ps1.') +param vmAdminPassword string = '' + +var tags = { + CostControl: 'Ignore' + SecurityControl: 'Ignore' + lab: 'offline-industrial-copilot' +} +var storageName = toLower('stoic${resourceSuffix}') +var keyVaultName = toLower('kv-oic-${resourceSuffix}') +var workspaceName = 'mlw-oic-${resourceSuffix}' +var vmName = 'vm-oic-${take(resourceSuffix, 8)}' + +// ---------- Azure Machine Learning workspace (Challenge 5: train in the cloud) ---------- + +resource storage 'Microsoft.Storage/storageAccounts@2023-05-01' = { + name: take(storageName, 24) + location: location + tags: tags + sku: { name: 'Standard_LRS' } + kind: 'StorageV2' + properties: { + minimumTlsVersion: 'TLS1_2' + allowBlobPublicAccess: false + } +} + +resource keyVault 'Microsoft.KeyVault/vaults@2023-07-01' = { + name: take(keyVaultName, 24) + location: location + tags: tags + properties: { + tenantId: subscription().tenantId + sku: { family: 'A', name: 'standard' } + enableRbacAuthorization: true + } +} + +resource logs 'Microsoft.OperationalInsights/workspaces@2023-09-01' = { + name: 'log-oic-${resourceSuffix}' + location: location + tags: tags + properties: { + sku: { name: 'PerGB2018' } + retentionInDays: 30 + } +} + +resource appInsights 'Microsoft.Insights/components@2020-02-02' = { + name: 'appi-oic-${resourceSuffix}' + location: location + tags: tags + kind: 'web' + properties: { + Application_Type: 'web' + WorkspaceResourceId: logs.id + } +} + +resource mlWorkspace 'Microsoft.MachineLearningServices/workspaces@2024-10-01' = { + name: workspaceName + location: location + tags: tags + identity: { type: 'SystemAssigned' } + properties: { + friendlyName: 'Offline Industrial Copilot' + storageAccount: storage.id + keyVault: keyVault.id + applicationInsights: appInsights.id + } +} + +// ---------- Optional Windows 11 VM (alternative to a laptop) ---------- + +resource nsg 'Microsoft.Network/networkSecurityGroups@2024-05-01' = if (deployVm) { + name: 'nsg-oic-${resourceSuffix}' + location: location + tags: tags + properties: { + securityRules: [ + { + name: 'allow-rdp' + properties: { + priority: 1000 + direction: 'Inbound' + access: 'Allow' + protocol: 'Tcp' + sourceAddressPrefix: '*' + sourcePortRange: '*' + destinationAddressPrefix: '*' + destinationPortRange: '3389' + } + } + ] + } +} + +resource vnet 'Microsoft.Network/virtualNetworks@2024-05-01' = if (deployVm) { + name: 'vnet-oic-${resourceSuffix}' + location: location + tags: tags + properties: { + addressSpace: { addressPrefixes: ['10.20.0.0/24'] } + subnets: [ + { + name: 'vm' + properties: { + addressPrefix: '10.20.0.0/24' + networkSecurityGroup: { id: nsg.id } + } + } + ] + } +} + +resource publicIp 'Microsoft.Network/publicIPAddresses@2024-05-01' = if (deployVm) { + name: 'pip-oic-${resourceSuffix}' + location: location + tags: tags + sku: { name: 'Standard' } + properties: { publicIPAllocationMethod: 'Static' } +} + +resource nic 'Microsoft.Network/networkInterfaces@2024-05-01' = if (deployVm) { + name: 'nic-oic-${resourceSuffix}' + location: location + tags: tags + properties: { + ipConfigurations: [ + { + name: 'ipconfig1' + properties: { + privateIPAllocationMethod: 'Dynamic' + subnet: { id: '${vnet.id}/subnets/vm' } + publicIPAddress: { id: publicIp.id } + } + } + ] + } +} + +resource vm 'Microsoft.Compute/virtualMachines@2024-07-01' = if (deployVm) { + name: vmName + location: location + tags: tags + properties: { + hardwareProfile: { vmSize: vmSize } + licenseType: 'Windows_Client' + osProfile: { + computerName: take(vmName, 15) + adminUsername: vmAdminUser + adminPassword: vmAdminPassword + } + storageProfile: { + imageReference: { + publisher: 'MicrosoftWindowsDesktop' + offer: 'windows-11' + sku: 'win11-24h2-pro' + version: 'latest' + } + osDisk: { + createOption: 'FromImage' + diskSizeGB: 128 + managedDisk: { storageAccountType: 'Premium_LRS' } + } + } + networkProfile: { networkInterfaces: [{ id: nic.id }] } + } +} + +// Installs Python for all users and the Visual C++ runtime that Foundry Local needs. A fresh Windows image does not +// have the runtime, and the Foundry Local service then stops at start-up with "Unable to load DLL 'foundry_local'". +resource vmSetup 'Microsoft.Compute/virtualMachines/extensions@2024-07-01' = if (deployVm) { + parent: vm + name: 'setup-python' + location: location + properties: { + publisher: 'Microsoft.Compute' + type: 'CustomScriptExtension' + typeHandlerVersion: '1.10' + autoUpgradeMinorVersion: true + protectedSettings: { + commandToExecute: 'powershell -NoProfile -ExecutionPolicy Bypass -Command "$ProgressPreference=\'SilentlyContinue\'; Invoke-WebRequest -Uri https://www.python.org/ftp/python/3.12.8/python-3.12.8-amd64.exe -OutFile C:\\py.exe; Start-Process C:\\py.exe -ArgumentList \'/quiet InstallAllUsers=1 PrependPath=1\' -Wait; Remove-Item C:\\py.exe; Invoke-WebRequest -Uri https://aka.ms/vs/17/release/vc_redist.x64.exe -OutFile C:\\vc.exe; Start-Process C:\\vc.exe -ArgumentList \'/install /quiet /norestart\' -Wait; Remove-Item C:\\vc.exe"' + } + } +} + +output mlWorkspaceName string = mlWorkspace.name +output vmName string = deployVm ? vmName : '' +output vmPublicIp string = deployVm ? publicIp!.properties.ipAddress : '' +output vmAdminUser string = deployVm ? vmAdminUser : '' diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/shared-deploy-lab.ps1 b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/shared-deploy-lab.ps1 new file mode 100644 index 000000000..54d9a38d5 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/labautomation/shared-deploy-lab.ps1 @@ -0,0 +1,65 @@ +<# +.SYNOPSIS +Runs once per subscription before any deploy-lab.ps1 run starts. +.DESCRIPTION +Registers the resource providers used by the Offline Industrial Copilot lab so the per-participant deployments do not race each other. +Also registers the Microsoft.Network/AllowBringYourOwnPublicIpAddress feature. Without it the subscription refuses every +public IP (SubscriptionNotRegisteredForFeature), and the lab VM needs one for Remote Desktop. +.PARAMETER SubscriptionId +Specifies the Azure subscription that contains the lab resources. +.PARAMETER PreferredLocation +Specifies the preferred Azure regions (ordered by preference) for resource deployment. +.PARAMETER AllowedEntraUserIds +Entra user object IDs of every participant holding a lab in this subscription. +#> +param( + [Parameter(Mandatory=$true)] + [string]$SubscriptionId, + + [Parameter(Mandatory=$true)] + [string[]]$PreferredLocation = @(), + + [Parameter(Mandatory=$false)] + [string[]]$AllowedEntraUserIds = @() +) + +$requiredProviders = @( + "Microsoft.Compute", + "Microsoft.Network", + "Microsoft.Storage", + "Microsoft.KeyVault", + "Microsoft.OperationalInsights", + "Microsoft.Insights", + "Microsoft.MachineLearningServices" +) +foreach($provider in $requiredProviders) { + $state = (Get-AzResourceProvider -ProviderNamespace $provider -ErrorAction SilentlyContinue | Select-Object -First 1).RegistrationState + if($state -ne "Registered") { + Write-Host "[$SubscriptionId] Registering resource provider: $provider" + Register-AzResourceProvider -ProviderNamespace $provider -ErrorAction Stop | Out-Null + } +} + +# The lab VM needs a public IP for Remote Desktop. Some subscriptions refuse every public IP until this feature is registered. +$feature = "AllowBringYourOwnPublicIpAddress" +$featureState = (Get-AzProviderFeature -ProviderNamespace "Microsoft.Network" -FeatureName $feature -ErrorAction Stop).RegistrationState +if($featureState -ne "Registered") { + Write-Host "[$SubscriptionId] Registering feature Microsoft.Network/$feature (state: $featureState)" + if($featureState -notin @("Registering", "Pending")) { + Register-AzProviderFeature -ProviderNamespace "Microsoft.Network" -FeatureName $feature -ErrorAction Stop | Out-Null + } + for($attempt = 1; $attempt -le 90 -and $featureState -ne "Registered"; $attempt++) { + Start-Sleep -Seconds 10 + $featureState = (Get-AzProviderFeature -ProviderNamespace "Microsoft.Network" -FeatureName $feature -ErrorAction Stop).RegistrationState + } + if($featureState -ne "Registered") { + throw "Microsoft.Network/$feature did not reach Registered within 15 minutes (state: $featureState). A subscription owner can run: az feature register --namespace Microsoft.Network --name $feature; az provider register --namespace Microsoft.Network" + } + # The feature only takes effect after Microsoft.Network is registered again. + Register-AzResourceProvider -ProviderNamespace "Microsoft.Network" -ErrorAction Stop | Out-Null + for($attempt = 1; $attempt -le 30; $attempt++) { + if((Get-AzResourceProvider -ProviderNamespace "Microsoft.Network" | Select-Object -First 1).RegistrationState -eq "Registered") { break } + Start-Sleep -Seconds 10 + } + Write-Host "[$SubscriptionId] Feature Microsoft.Network/$feature is registered" +} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/compound_eval.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/compound_eval.json new file mode 100644 index 000000000..f0eb3c1f0 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/compound_eval.json @@ -0,0 +1,5 @@ +[ + {"id": "C01", "question": "What torque do I use for the spindle bearing cartridge bolts, and what must I do before I start work on the spindle?", "expected_sops": ["SOP-CNC-001", "SOP-SAF-001"], "must_include": ["45 Nm", "lock"]}, + {"id": "C02", "question": "How often is the hydraulic press oil changed, and what is the first response time for a high priority ticket?", "expected_sops": ["SOP-HYD-001", "SOP-GEN-001"], "must_include": ["4000", "1 hour"]}, + {"id": "C03", "question": "What coolant mixing ratio do the CNC machines use, and how often is the compressor oil replaced?", "expected_sops": ["SOP-CNC-002", "SOP-CMP-001"], "must_include": ["8 percent", "2000"]} +] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/retrieval_eval.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/retrieval_eval.json new file mode 100644 index 000000000..476a8e08b --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/retrieval_eval.json @@ -0,0 +1,17 @@ +[ + {"id": "Q01", "question": "At what vibration level must a CNC spindle be stopped?", "expected_sop": "SOP-CNC-001", "must_include": ["7.1"]}, + {"id": "Q02", "question": "What torque do I use for the spindle bearing cartridge bolts?", "expected_sop": "SOP-CNC-001", "must_include": ["45 Nm"]}, + {"id": "Q03", "question": "Who is allowed to replace the spindle bearing?", "expected_sop": "SOP-CNC-001", "must_include": ["spindle-certified"]}, + {"id": "Q04", "question": "What does error E-CLT-042 mean and what do I do first?", "expected_sop": "SOP-CNC-002", "must_include": ["coolant", "level"]}, + {"id": "Q05", "question": "What coolant mixing ratio should the CNC machines use?", "expected_sop": "SOP-CNC-002", "must_include": ["8 percent"]}, + {"id": "Q06", "question": "Which oil does the hydraulic press use and how often is it changed?", "expected_sop": "SOP-HYD-001", "must_include": ["ISO VG 46", "4000"]}, + {"id": "Q07", "question": "At what oil temperature does the hydraulic press shut down?", "expected_sop": "SOP-HYD-001", "must_include": ["75"]}, + {"id": "Q08", "question": "How often must the compressor oil be replaced?", "expected_sop": "SOP-CMP-001", "must_include": ["2000"]}, + {"id": "Q09", "question": "The compressor shows E-CMP-107. What is the temperature limit?", "expected_sop": "SOP-CMP-001", "must_include": ["100"]}, + {"id": "Q10", "question": "What motor current triggers the conveyor overload error?", "expected_sop": "SOP-CNV-001", "must_include": ["14 A"]}, + {"id": "Q11", "question": "How often should the welding contact tip be replaced?", "expected_sop": "SOP-ROB-001", "must_include": ["8 hours"]}, + {"id": "Q12", "question": "What is the first response time for a high-priority ticket?", "expected_sop": "SOP-GEN-001", "must_include": ["1 hour"]}, + {"id": "Q13", "question": "What do I do before I open any machine cover for maintenance?", "expected_sop": "SOP-SAF-001", "must_include": ["lock"]}, + {"id": "U01", "question": "What is the cafeteria menu on Friday?", "expected_sop": null, "must_include": [], "unanswerable": true}, + {"id": "U02", "question": "What torque do I use for the wheel bolts on the forklift?", "expected_sop": null, "must_include": [], "unanswerable": true} +] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/routing_eval.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/routing_eval.json new file mode 100644 index 000000000..cd6b2fcc6 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/routing_eval.json @@ -0,0 +1,37 @@ +[ + {"id": "S01", "request": "What is the status of machine CNC-07?", "expected_tool": "get_machine_status"}, + {"id": "S02", "request": "Is PRESS-02 running OK right now?", "expected_tool": "get_machine_status"}, + {"id": "S03", "request": "Give me the current readings from the compressor CMP-01.", "expected_tool": "get_machine_status"}, + {"id": "S04", "request": "How hot is the welding robot ROB-04 at the moment?", "expected_tool": "get_machine_status"}, + {"id": "S05", "request": "Check the conveyor CONV-03 for me.", "expected_tool": "get_machine_status"}, + {"id": "S06", "request": "What is the vibration on LATHE-05 right now?", "expected_tool": "get_machine_status"}, + {"id": "S07", "request": "Is CNC-03 running or stopped?", "expected_tool": "get_machine_status"}, + {"id": "S08", "request": "Show me the load and current of CNC-07.", "expected_tool": "get_machine_status"}, + + {"id": "F01", "request": "Predict the most likely fault for CNC-07 from its sensor readings.", "expected_tool": "predict_fault"}, + {"id": "F02", "request": "Run the fault prediction model for CNC-07.", "expected_tool": "predict_fault"}, + {"id": "F03", "request": "Use the fault classifier to predict the fault of CNC-07.", "expected_tool": "predict_fault"}, + {"id": "F04", "request": "What does the classifier say is wrong with CMP-01?", "expected_tool": "predict_fault"}, + {"id": "F05", "request": "Diagnose PRESS-02.", "expected_tool": "predict_fault"}, + {"id": "F06", "request": "Which kind of failure is LATHE-05 heading for?", "expected_tool": "predict_fault"}, + {"id": "F07", "request": "According to the model, what is the likely cause of the problem on CNC-07?", "expected_tool": "predict_fault"}, + {"id": "F08", "request": "Is CMP-01 more likely overheating or low on lubrication?", "expected_tool": "predict_fault"}, + + {"id": "P01", "request": "At what vibration level must a CNC spindle be stopped?", "expected_tool": "search_sops"}, + {"id": "P02", "request": "What torque do I use for the spindle bearing cartridge bolts?", "expected_tool": "search_sops"}, + {"id": "P03", "request": "How often must the compressor oil be replaced?", "expected_tool": "search_sops"}, + {"id": "P04", "request": "What do I do before I open any machine cover?", "expected_tool": "search_sops"}, + {"id": "P05", "request": "What does error E-CLT-042 mean?", "expected_tool": "search_sops"}, + {"id": "P06", "request": "Which oil does the hydraulic press use?", "expected_tool": "search_sops"}, + {"id": "P07", "request": "What is the first response time for a high priority ticket?", "expected_tool": "search_sops"}, + {"id": "P08", "request": "Why is the sky blue?", "expected_tool": "search_sops"}, + {"id": "P09", "request": "What does the SOP say about the vibration limit for CNC-07?", "expected_tool": "search_sops"}, + {"id": "P10", "request": "CNC-07 shows error E-SPN-214. What should I do about it?", "expected_tool": "search_sops"}, + + {"id": "T01", "request": "Open a high priority ticket for CNC-07: spindle vibration above the stop limit.", "expected_tool": "create_maintenance_ticket"}, + {"id": "T02", "request": "Please create a low priority ticket for ROB-04: nozzle needs cleaning at the next planned stop.", "expected_tool": "create_maintenance_ticket"}, + {"id": "T03", "request": "Log a medium priority maintenance ticket for LATHE-05, vibration is close to the warning limit.", "expected_tool": "create_maintenance_ticket"}, + {"id": "T04", "request": "Raise a ticket for PRESS-02, the oil is running too hot.", "expected_tool": "create_maintenance_ticket"}, + {"id": "T05", "request": "I need a ticket for CMP-01, it keeps tripping.", "expected_tool": "create_maintenance_ticket"}, + {"id": "T06", "request": "Can you file a work order for CNC-03? The coolant is low.", "expected_tool": "create_maintenance_ticket"} +] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/routing_holdout.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/routing_holdout.json new file mode 100644 index 000000000..a49a42242 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/routing_holdout.json @@ -0,0 +1,28 @@ +[ + {"id": "H01", "request": "Tell me how LATHE-05 is doing.", "expected_tool": "get_machine_status"}, + {"id": "H02", "request": "What temperature is CNC-03 at?", "expected_tool": "get_machine_status"}, + {"id": "H03", "request": "Can you read out the numbers for PRESS-02?", "expected_tool": "get_machine_status"}, + {"id": "H04", "request": "Is CONV-03 up and running?", "expected_tool": "get_machine_status"}, + {"id": "H05", "request": "What is the current draw on CMP-01?", "expected_tool": "get_machine_status"}, + + {"id": "H06", "request": "Which fault does the model expect on ROB-04?", "expected_tool": "predict_fault"}, + {"id": "H07", "request": "Run a diagnosis on CNC-07.", "expected_tool": "predict_fault"}, + {"id": "H08", "request": "Can you tell what is likely breaking on LATHE-05?", "expected_tool": "predict_fault"}, + {"id": "H09", "request": "Classify the condition of PRESS-02.", "expected_tool": "predict_fault"}, + {"id": "H10", "request": "What failure mode do the readings of CMP-01 point to?", "expected_tool": "predict_fault"}, + + {"id": "H11", "request": "How tight should the coolant pump bolts be?", "expected_tool": "search_sops"}, + {"id": "H12", "request": "When must the conveyor bearings be greased?", "expected_tool": "search_sops"}, + {"id": "H13", "request": "What is the correct procedure for a lockout?", "expected_tool": "search_sops"}, + {"id": "H14", "request": "What does the controller code E-HYD-031 stand for?", "expected_tool": "search_sops"}, + {"id": "H15", "request": "PRESS-02 is showing E-HYD-031. How do I fix it?", "expected_tool": "search_sops"}, + {"id": "H16", "request": "What is the temperature limit before the compressor trips?", "expected_tool": "search_sops"}, + {"id": "H17", "request": "Who is allowed to replace a spindle bearing?", "expected_tool": "search_sops"}, + {"id": "H18", "request": "What is the recommended mixing ratio of the coolant?", "expected_tool": "search_sops"}, + + {"id": "H19", "request": "Make a ticket for CNC-07: strange noise from the spindle.", "expected_tool": "create_maintenance_ticket"}, + {"id": "H20", "request": "Write up a high priority work order for CMP-01, it overheats.", "expected_tool": "create_maintenance_ticket"}, + {"id": "H21", "request": "Please log a ticket against ROB-04 for the worn contact tip.", "expected_tool": "create_maintenance_ticket"}, + {"id": "H22", "request": "Create a service request for CONV-03, the belt is drifting.", "expected_tool": "create_maintenance_ticket"}, + {"id": "H23", "request": "Open a low priority ticket for LATHE-05.", "expected_tool": "create_maintenance_ticket"} +] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/tool_eval.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/tool_eval.json new file mode 100644 index 000000000..4403068d9 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/eval/tool_eval.json @@ -0,0 +1,10 @@ +[ + {"id": "T01", "prompt": "What is the status of machine CNC-07?", "expected_tool": "get_machine_status", "expected_args": {"machine_id": "CNC-07"}}, + {"id": "T02", "prompt": "Is PRESS-02 running OK right now?", "expected_tool": "get_machine_status", "expected_args": {"machine_id": "PRESS-02"}}, + {"id": "T03", "prompt": "Check the conveyor CONV-03 for me.", "expected_tool": "get_machine_status", "expected_args": {"machine_id": "CONV-03"}}, + {"id": "T04", "prompt": "Give me the current reading from the compressor CMP-01.", "expected_tool": "get_machine_status", "expected_args": {"machine_id": "CMP-01"}}, + {"id": "T05", "prompt": "Open a high priority ticket for CNC-07: spindle vibration above the stop limit.", "expected_tool": "create_maintenance_ticket", "expected_args": {"machine_id": "CNC-07", "priority": "high"}}, + {"id": "T06", "prompt": "Please create a low priority ticket for ROB-04: nozzle needs cleaning at next planned stop.", "expected_tool": "create_maintenance_ticket", "expected_args": {"machine_id": "ROB-04", "priority": "low"}}, + {"id": "T07", "prompt": "Log a medium priority maintenance ticket for LATHE-05, vibration is close to the warning limit.", "expected_tool": "create_maintenance_ticket", "expected_args": {"machine_id": "LATHE-05", "priority": "medium"}}, + {"id": "T08", "prompt": "Why is the sky blue?", "expected_tool": null, "expected_args": {}} +] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/machines.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/machines.json new file mode 100644 index 000000000..2238a691f --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/machines.json @@ -0,0 +1,104 @@ +{ + "plant": "Contoso Precision Machining", + "as_of": "2026-10-20T06:30:00Z", + "machines": [ + { + "id": "CNC-07", + "name": "5-axis CNC mill", + "line": "A", + "state": "FAULT", + "error_code": "E-SPN-214", + "error_text": "Spindle vibration high", + "vibration_mm_s": 9.4, + "temperature_c": 78.0, + "load_pct": 82, + "current_a": 31.5, + "hours_since_service": 1840, + "last_maintenance": "2026-06-12" + }, + { + "id": "CNC-03", + "name": "3-axis CNC mill", + "line": "A", + "state": "RUNNING", + "error_code": null, + "error_text": null, + "vibration_mm_s": 2.1, + "temperature_c": 61.0, + "load_pct": 55, + "current_a": 22.4, + "hours_since_service": 420, + "last_maintenance": "2026-09-02" + }, + { + "id": "PRESS-02", + "name": "Hydraulic press", + "line": "B", + "state": "WARNING", + "error_code": "E-HYD-031", + "error_text": "Oil temperature high", + "vibration_mm_s": 3.2, + "temperature_c": 71.0, + "load_pct": 74, + "current_a": 44.0, + "hours_since_service": 3650, + "last_maintenance": "2026-02-18" + }, + { + "id": "CONV-03", + "name": "Belt conveyor", + "line": "B", + "state": "RUNNING", + "error_code": null, + "error_text": null, + "vibration_mm_s": 1.4, + "temperature_c": 44.0, + "load_pct": 48, + "current_a": 11.8, + "hours_since_service": 470, + "last_maintenance": "2026-09-10" + }, + { + "id": "CMP-01", + "name": "Air compressor", + "line": "Utilities", + "state": "WARNING", + "error_code": "E-CMP-107", + "error_text": "Discharge temperature high", + "vibration_mm_s": 2.8, + "temperature_c": 104.0, + "load_pct": 91, + "current_a": 58.0, + "hours_since_service": 1950, + "last_maintenance": "2026-03-30" + }, + { + "id": "ROB-04", + "name": "Welding robot", + "line": "C", + "state": "IDLE", + "error_code": null, + "error_text": null, + "vibration_mm_s": 0.6, + "temperature_c": 31.0, + "load_pct": 0, + "current_a": 1.2, + "hours_since_service": 160, + "last_maintenance": "2026-10-01" + }, + { + "id": "LATHE-05", + "name": "CNC lathe", + "line": "A", + "state": "RUNNING", + "error_code": null, + "error_text": null, + "vibration_mm_s": 4.9, + "temperature_c": 67.0, + "load_pct": 63, + "current_a": 26.0, + "hours_since_service": 1210, + "last_maintenance": "2026-07-22" + } + ] +} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sensor_history.csv b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sensor_history.csv new file mode 100644 index 000000000..ea28aea36 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sensor_history.csv @@ -0,0 +1,2401 @@ +vibration_mm_s,temperature_c,load_pct,current_a,hours_since_service,label +6.2,68.7,33.1,14.1,2479.3,low_lubrication +12.3,69.6,68.9,24.5,2614.9,bearing_wear +4.1,70.6,69.7,21.0,3692.3,low_lubrication +5.1,62.8,82.6,27.3,3572.5,low_lubrication +3.8,87.9,38.9,22.0,659.9,overheating +0.3,52.1,71.8,27.5,2070.5,normal +1.5,65.1,35.7,13.0,1795.9,normal +2.3,47.2,40.3,15.3,649.4,normal +3.4,55.2,32.2,9.1,1919.2,normal +5.7,70.8,36.9,14.0,2899.5,low_lubrication +2.6,62.8,33.0,12.6,348.7,normal +5.4,75.0,73.7,26.6,2294.0,low_lubrication +2.9,97.2,83.0,36.2,1884.9,overheating +2.1,63.1,70.8,23.1,1331.9,normal +3.1,54.0,68.1,24.5,1005.3,normal +2.9,56.8,78.0,30.3,1407.8,normal +2.2,62.0,64.1,24.2,2039.3,normal +2.5,64.0,33.5,13.7,1407.5,normal +5.6,72.2,51.0,16.1,2837.5,low_lubrication +2.2,57.1,78.8,25.6,1605.8,normal +1.9,52.5,74.6,29.2,709.0,normal +7.6,77.9,56.0,18.6,3343.2,bearing_wear +2.3,61.6,46.9,19.5,486.6,normal +7.8,74.8,59.7,20.5,2394.6,bearing_wear +3.4,87.8,79.1,35.7,2955.6,overheating +3.9,95.1,85.0,35.3,1710.6,overheating +7.2,66.5,59.0,20.1,1426.2,bearing_wear +0.3,99.9,57.0,28.7,949.8,overheating +3.1,50.3,36.4,13.1,1607.9,normal +2.4,93.2,44.8,19.2,2383.7,overheating +9.1,76.6,59.7,16.2,1904.4,bearing_wear +3.6,60.9,76.4,27.9,1729.6,normal +10.8,74.6,59.1,23.6,1599.8,bearing_wear +5.7,76.6,46.4,15.2,2036.8,bearing_wear +1.7,59.1,41.5,17.9,1772.4,normal +2.3,60.2,36.9,13.7,1931.3,normal +6.0,66.1,35.7,14.5,3761.6,low_lubrication +2.6,93.6,89.5,38.1,2940.6,overheating +5.8,69.7,72.3,24.6,2175.8,low_lubrication +5.4,59.0,40.6,15.0,3540.4,low_lubrication +3.5,100.6,36.7,18.4,3476.8,overheating +4.7,92.1,42.2,23.2,906.2,overheating +6.1,66.0,62.0,23.5,4094.1,low_lubrication +2.3,59.8,32.7,8.7,695.5,normal +4.6,94.6,64.5,30.6,568.0,overheating +9.7,62.8,63.1,23.0,1807.5,bearing_wear +3.1,55.4,52.0,16.8,1182.2,normal +5.3,73.2,41.4,15.3,2723.2,bearing_wear +3.5,67.9,48.8,19.0,2676.9,low_lubrication +2.4,42.8,30.8,10.0,1241.4,normal +5.2,63.2,72.6,25.9,2128.0,low_lubrication +2.2,61.9,50.3,16.7,525.2,normal +6.6,69.4,81.8,29.8,1898.4,bearing_wear +5.2,86.0,68.9,29.9,2297.2,overheating +6.2,69.4,84.8,29.9,3170.0,low_lubrication +5.2,73.1,82.5,33.7,2393.3,bearing_wear +3.4,94.4,77.0,33.9,1259.6,overheating +6.7,71.2,84.5,31.6,1873.7,bearing_wear +6.3,69.0,55.2,19.7,2481.9,low_lubrication +7.4,77.8,86.7,30.9,3021.2,bearing_wear +6.2,70.0,38.7,14.7,2989.0,low_lubrication +7.5,69.0,81.5,27.6,2181.5,bearing_wear +2.5,67.4,38.6,14.2,206.1,normal +9.3,75.7,88.3,31.0,2429.2,bearing_wear +7.5,68.8,31.5,9.3,3425.1,bearing_wear +5.3,67.4,55.6,22.2,2662.7,low_lubrication +9.1,66.9,46.5,16.9,3041.0,bearing_wear +9.0,77.4,59.3,22.0,1869.5,bearing_wear +5.0,74.2,42.0,15.7,2862.6,low_lubrication +6.3,59.8,32.6,10.5,1423.6,bearing_wear +5.4,57.3,74.8,24.3,3130.9,low_lubrication +6.8,69.8,87.5,31.6,2253.3,bearing_wear +3.0,99.3,31.4,22.8,879.6,overheating +3.4,76.7,61.5,24.9,1022.6,overheating +8.7,73.9,39.3,15.7,2352.0,bearing_wear +0.9,62.3,74.8,27.9,1235.6,normal +3.7,56.5,63.3,20.0,1132.2,normal +2.5,102.6,46.0,21.3,1458.3,overheating +6.3,70.4,74.7,26.1,2468.9,low_lubrication +0.2,62.0,55.0,18.5,1766.7,normal +5.2,63.9,67.4,24.4,2056.1,low_lubrication +2.7,98.0,72.0,32.9,2616.4,overheating +6.3,70.1,49.6,20.2,3376.2,low_lubrication +3.8,88.8,34.7,16.1,595.7,overheating +5.0,69.8,59.3,18.8,2760.1,low_lubrication +5.9,65.7,49.5,14.9,2222.5,low_lubrication +5.7,58.2,44.1,14.5,2992.8,low_lubrication +3.8,59.9,57.3,21.0,943.2,normal +4.3,79.1,74.3,33.0,1216.7,overheating +4.8,60.9,81.6,27.5,3305.0,low_lubrication +2.4,65.5,69.1,19.7,2044.6,normal +5.2,101.4,55.6,24.4,3405.5,overheating +5.9,70.7,38.4,14.7,2466.1,bearing_wear +7.7,79.8,43.5,18.9,2453.6,bearing_wear +5.6,75.8,66.6,23.6,2277.6,low_lubrication +3.0,54.2,59.5,20.7,1729.7,normal +6.3,69.2,72.2,22.3,3779.5,low_lubrication +5.9,59.2,61.8,20.7,2829.6,low_lubrication +3.4,103.3,54.7,27.4,2484.9,overheating +5.3,67.5,73.5,26.4,2715.4,low_lubrication +4.9,66.6,34.4,11.1,3859.0,low_lubrication +2.4,55.4,64.3,21.0,39.5,normal +7.9,69.2,56.5,22.9,1371.8,bearing_wear +10.4,73.6,77.4,29.4,3372.3,bearing_wear +6.1,70.7,49.2,19.9,2660.8,bearing_wear +5.5,61.2,31.5,11.1,3359.3,low_lubrication +3.6,81.2,75.0,30.4,1675.8,overheating +8.4,78.9,52.6,21.4,2084.5,bearing_wear +3.2,54.4,75.5,26.9,1010.7,normal +5.7,86.6,55.4,24.3,2361.9,overheating +6.4,71.3,68.2,26.8,3589.3,low_lubrication +7.3,66.7,65.3,20.6,1911.7,bearing_wear +5.7,67.5,87.5,33.9,3460.5,low_lubrication +2.0,97.8,85.7,33.5,3470.7,overheating +5.6,66.8,31.7,10.6,2591.1,low_lubrication +2.6,90.2,30.2,18.5,3478.5,overheating +3.9,98.5,60.1,27.7,1966.9,overheating +3.0,46.0,87.5,30.6,2283.5,normal +5.1,68.0,30.3,8.0,3497.4,bearing_wear +2.2,90.6,83.1,35.1,1969.1,overheating +5.2,77.5,38.3,15.6,1965.6,bearing_wear +2.8,61.0,64.8,25.4,1268.0,normal +1.6,67.1,86.3,28.1,2488.9,normal +4.0,85.4,67.0,29.0,3096.5,overheating +4.5,82.9,86.7,35.9,3418.9,overheating +2.9,56.5,38.9,12.3,153.3,normal +9.4,72.5,35.4,13.2,1833.7,bearing_wear +8.3,69.9,50.8,18.4,2498.0,bearing_wear +4.1,65.2,81.0,28.7,2858.6,low_lubrication +3.3,84.2,75.5,29.0,2928.1,overheating +7.5,76.2,74.2,29.2,1751.3,bearing_wear +2.9,94.3,32.1,18.7,1550.5,overheating +3.4,88.0,32.6,17.0,3114.5,overheating +6.4,64.2,76.6,26.8,3621.4,low_lubrication +4.1,88.3,53.5,22.5,2735.6,overheating +2.2,55.3,85.0,28.6,2223.3,normal +2.5,58.9,35.0,14.3,1372.2,normal +5.2,74.7,73.5,27.0,3909.2,low_lubrication +2.3,53.0,81.0,34.8,706.3,normal +9.5,70.8,58.3,22.8,2069.4,bearing_wear +2.7,68.9,51.3,17.3,1457.4,normal +1.4,52.9,53.9,22.5,732.3,normal +2.9,62.0,58.7,20.8,2197.7,low_lubrication +6.2,60.3,74.4,26.3,2150.2,low_lubrication +7.7,68.1,44.9,13.5,2298.9,bearing_wear +3.4,57.0,80.7,32.0,2448.1,normal +7.9,81.8,32.6,11.8,1494.8,bearing_wear +3.4,49.6,52.3,17.8,2394.2,normal +2.9,92.8,73.5,29.7,1037.1,overheating +5.9,66.2,47.6,18.5,3701.4,low_lubrication +4.3,59.9,84.6,30.9,3115.5,low_lubrication +9.9,56.8,63.8,26.1,1474.7,bearing_wear +3.8,51.5,31.7,9.6,1082.5,normal +3.3,109.5,87.9,34.4,2789.6,overheating +6.0,69.4,61.5,22.9,2797.0,low_lubrication +2.1,67.6,78.1,28.3,2037.6,normal +1.5,57.8,86.2,28.9,1489.6,normal +3.7,97.0,35.8,20.1,2073.7,overheating +1.6,58.9,58.5,20.9,1557.7,normal +2.4,60.4,31.8,7.4,1305.4,normal +5.3,69.3,35.0,13.8,4069.8,low_lubrication +2.3,88.1,88.0,36.8,1047.9,overheating +6.5,65.8,42.0,16.6,3700.3,low_lubrication +4.0,81.6,69.6,25.6,1034.1,overheating +4.8,68.7,34.8,14.0,3840.2,low_lubrication +0.8,56.8,79.8,24.4,311.0,normal +2.9,49.4,54.5,20.1,144.0,normal +1.9,52.8,80.0,29.8,1478.4,normal +2.4,55.8,69.1,24.0,1772.2,normal +8.7,70.5,30.7,12.5,2912.7,bearing_wear +1.2,82.7,72.9,33.0,696.6,overheating +5.4,61.5,70.5,22.6,2384.5,low_lubrication +4.9,96.7,72.6,33.2,2202.0,overheating +2.0,60.9,41.0,12.2,1374.2,normal +9.1,79.7,81.0,28.0,3131.1,bearing_wear +6.3,70.2,54.6,17.6,2567.5,bearing_wear +2.1,88.1,57.7,28.9,1760.1,overheating +2.7,66.0,79.8,27.4,175.3,normal +10.0,71.4,74.4,26.4,3210.2,bearing_wear +6.6,75.3,36.1,12.9,1322.3,bearing_wear +3.0,90.2,35.5,19.8,907.3,overheating +3.4,90.5,58.3,27.9,2619.2,overheating +8.7,77.8,84.9,29.0,1548.9,bearing_wear +4.7,57.7,55.7,15.3,2492.7,normal +6.5,69.5,87.1,31.3,2242.6,bearing_wear +5.9,79.0,60.7,20.1,2307.9,low_lubrication +1.7,57.7,74.9,28.8,1535.5,normal +5.5,63.7,63.0,24.5,3682.7,low_lubrication +1.3,68.8,60.0,22.6,603.3,normal +3.3,98.5,37.3,18.8,1465.1,overheating +3.8,90.7,38.2,22.9,1407.0,overheating +1.9,91.2,35.0,16.3,2151.5,overheating +5.4,71.1,62.4,21.6,3719.2,low_lubrication +3.4,95.7,61.9,27.5,1741.7,overheating +3.1,67.1,88.8,30.4,623.9,normal +2.3,91.3,43.0,18.1,1060.4,overheating +5.7,95.8,38.2,21.2,3127.5,overheating +5.1,67.4,89.7,31.4,3615.5,low_lubrication +1.5,58.1,47.2,15.7,1050.4,normal +2.4,88.4,45.0,22.2,3204.4,overheating +4.0,57.7,68.3,27.5,1977.2,normal +3.0,55.6,47.5,15.3,1017.1,normal +2.3,47.5,32.5,13.8,1248.2,normal +3.2,91.9,75.5,35.7,3169.2,overheating +5.1,72.5,80.0,27.9,3123.7,low_lubrication +6.1,76.2,53.6,20.0,2788.6,bearing_wear +3.7,56.2,53.0,20.3,611.9,normal +4.6,98.6,81.0,33.4,2169.9,overheating +1.9,65.9,52.4,18.3,2151.4,normal +3.1,92.1,68.3,30.0,3296.8,overheating +6.2,75.4,31.0,8.5,2276.6,bearing_wear +7.5,73.5,87.2,31.3,2935.2,bearing_wear +3.7,74.3,71.5,26.1,3389.3,low_lubrication +2.3,53.5,71.7,24.3,523.2,normal +4.2,99.1,71.4,31.4,2818.5,overheating +2.8,56.2,63.4,21.4,133.3,normal +7.8,70.8,33.8,11.7,3386.4,bearing_wear +1.7,100.0,68.7,34.2,1635.2,overheating +3.9,89.5,89.5,39.6,2016.4,overheating +3.6,80.6,56.0,17.0,3238.7,bearing_wear +6.7,64.2,66.1,28.6,4163.5,low_lubrication +4.8,72.5,85.7,30.3,3367.8,low_lubrication +3.0,99.6,75.8,34.0,2477.8,overheating +7.7,80.4,86.6,29.3,3011.8,bearing_wear +5.1,93.9,42.6,22.1,3409.2,overheating +4.3,62.1,56.1,18.9,1883.7,normal +8.9,65.8,74.9,25.1,3038.8,bearing_wear +7.7,79.0,82.1,26.1,1604.9,bearing_wear +3.3,105.0,55.2,27.6,1548.8,overheating +3.1,57.9,64.9,24.2,77.5,normal +5.4,91.5,46.4,25.3,2409.6,overheating +5.1,79.1,53.6,25.1,1887.3,overheating +5.4,63.7,81.0,28.2,3017.7,low_lubrication +4.0,58.3,43.5,17.5,3049.1,low_lubrication +4.0,97.9,66.0,30.9,1525.7,overheating +2.7,93.7,61.3,26.0,1773.5,overheating +1.6,63.4,30.4,9.2,933.5,normal +6.9,66.8,61.4,20.3,4063.7,low_lubrication +0.2,64.5,57.2,19.6,853.4,normal +6.0,68.9,30.6,13.4,3622.3,low_lubrication +4.1,95.7,31.1,18.7,2290.7,overheating +7.8,71.4,79.3,27.8,3191.9,bearing_wear +5.2,69.1,39.4,12.1,2891.2,low_lubrication +7.9,67.2,72.9,24.0,2480.1,bearing_wear +7.5,74.9,66.9,25.1,1263.6,bearing_wear +8.2,62.4,65.4,25.7,1666.7,bearing_wear +4.4,70.1,66.5,25.0,2502.6,low_lubrication +3.0,63.7,58.3,23.9,277.1,normal +3.3,84.4,72.9,29.7,1408.4,overheating +2.5,62.3,83.0,34.3,1322.6,normal +3.3,107.1,40.0,19.6,2581.2,overheating +2.7,62.5,78.3,30.6,1808.3,normal +3.7,58.1,87.5,31.9,2102.8,normal +2.5,50.6,50.0,20.3,556.2,normal +6.1,77.6,88.9,33.9,3263.8,bearing_wear +3.1,65.2,52.4,18.4,1932.2,normal +2.5,62.4,87.5,25.7,1291.4,normal +2.5,50.5,65.3,22.3,2308.3,normal +7.3,81.1,90.0,34.2,1355.6,bearing_wear +4.9,92.4,80.8,35.5,2582.2,overheating +6.4,67.5,71.0,29.8,3909.2,low_lubrication +8.4,83.8,67.2,24.2,2475.2,bearing_wear +5.8,69.3,30.3,9.1,3390.1,low_lubrication +6.2,56.5,85.7,25.6,2542.5,low_lubrication +2.8,90.8,45.2,20.5,2072.0,overheating +7.1,66.7,32.2,10.1,2848.0,bearing_wear +6.3,63.9,80.6,27.2,2251.8,low_lubrication +5.1,69.7,37.2,13.6,2118.8,low_lubrication +5.3,68.7,74.2,27.8,3842.3,low_lubrication +5.1,65.4,36.0,11.0,2460.3,low_lubrication +3.5,89.9,46.5,25.9,1700.2,overheating +2.7,88.7,48.2,23.4,1625.7,overheating +5.1,63.2,56.9,19.4,3268.2,low_lubrication +2.0,57.1,76.7,28.5,1800.5,normal +4.8,96.5,74.6,31.4,1394.4,overheating +6.2,72.7,38.6,12.6,1812.2,bearing_wear +1.2,55.7,64.3,26.6,618.1,normal +1.3,57.3,48.7,21.4,392.6,normal +2.1,93.7,43.3,22.0,1594.5,overheating +6.5,71.6,65.5,22.1,3452.1,low_lubrication +3.2,60.0,38.7,12.1,270.8,normal +1.8,61.5,73.2,25.8,1318.9,normal +2.1,88.8,80.9,37.6,1551.8,overheating +8.3,78.2,72.7,27.2,2611.5,bearing_wear +7.1,63.6,75.3,23.9,1373.2,bearing_wear +7.6,71.4,70.1,25.6,2319.5,bearing_wear +8.7,67.7,83.0,29.5,2179.5,bearing_wear +4.4,100.6,53.2,26.1,2724.0,overheating +5.4,64.7,65.4,25.8,3574.6,low_lubrication +4.0,99.7,31.9,17.5,1458.2,overheating +9.3,68.9,41.1,18.4,1259.6,bearing_wear +3.8,95.9,51.5,22.7,2001.9,overheating +5.8,62.4,85.1,32.4,2455.9,bearing_wear +7.5,70.3,77.9,30.3,3076.6,bearing_wear +8.9,73.6,55.0,21.5,1386.9,bearing_wear +6.8,73.0,87.9,29.5,2127.2,bearing_wear +5.7,74.9,35.7,11.8,2908.9,low_lubrication +3.4,93.6,65.0,29.9,1934.5,overheating +4.8,97.3,73.9,33.5,2003.4,overheating +2.1,64.9,48.4,18.5,1274.3,normal +3.1,92.8,57.2,27.7,3300.2,overheating +3.0,83.5,80.3,33.6,618.9,overheating +8.6,73.1,38.6,18.1,2168.9,bearing_wear +2.6,53.2,45.7,16.9,493.7,normal +1.0,64.5,63.3,22.2,1348.6,normal +8.9,79.5,71.1,28.8,1770.8,bearing_wear +1.7,67.1,87.3,30.5,2482.7,normal +4.6,72.9,56.2,22.7,3663.9,low_lubrication +7.6,81.4,70.4,23.7,1543.3,bearing_wear +2.2,66.8,33.3,12.4,542.0,normal +3.4,86.2,56.1,28.0,2980.9,overheating +5.7,74.0,80.7,27.7,3155.3,low_lubrication +1.6,45.1,36.2,10.7,1731.9,normal +8.7,69.4,78.1,27.1,3251.7,bearing_wear +1.3,67.5,44.5,15.1,2057.7,normal +3.4,93.1,43.2,19.5,507.3,overheating +7.3,74.6,61.7,16.5,2845.8,bearing_wear +1.1,59.1,89.8,32.8,1128.0,normal +9.5,85.0,33.9,9.7,1677.8,bearing_wear +4.1,58.5,43.6,16.3,506.0,normal +3.1,101.0,71.5,32.9,2136.1,overheating +3.9,86.8,86.3,35.4,2324.7,overheating +4.7,68.5,78.5,26.2,2711.9,low_lubrication +2.2,56.8,89.3,33.0,708.7,normal +4.1,101.4,39.7,18.8,2854.9,overheating +7.9,81.3,62.8,21.7,1989.2,bearing_wear +0.8,61.1,54.8,17.0,1455.0,normal +6.0,74.5,49.1,18.5,2948.6,bearing_wear +4.2,95.3,69.1,32.1,1489.0,overheating +7.0,66.4,41.1,18.7,2346.9,bearing_wear +2.5,70.3,50.4,18.5,1475.0,normal +1.9,58.7,62.0,21.7,2205.2,normal +6.6,79.0,34.6,15.7,2597.0,bearing_wear +5.4,92.5,57.1,24.4,3271.0,overheating +3.9,91.6,53.7,28.1,3059.7,overheating +3.4,89.4,73.7,30.5,3308.3,overheating +6.3,75.8,41.3,11.5,1332.5,bearing_wear +2.3,91.5,66.7,32.0,1529.2,overheating +5.7,59.6,53.0,17.7,2845.7,low_lubrication +4.0,55.2,46.9,16.3,2376.2,normal +6.4,84.0,70.9,27.5,1998.2,bearing_wear +4.1,59.0,44.4,14.0,1095.6,normal +4.8,53.3,72.3,26.1,587.1,normal +2.7,60.6,37.3,12.9,1071.5,normal +8.5,81.0,85.5,32.5,2042.2,bearing_wear +7.3,74.9,78.8,25.5,1704.8,bearing_wear +3.6,63.9,82.0,29.9,1864.1,normal +5.1,96.0,75.8,34.5,2325.7,overheating +3.9,59.2,61.5,22.2,3128.9,low_lubrication +8.9,66.2,83.2,28.0,1745.4,bearing_wear +6.5,69.8,51.2,15.1,2765.9,bearing_wear +2.9,85.3,82.5,36.9,1596.3,overheating +1.6,91.2,69.0,28.7,2252.2,overheating +6.1,60.3,77.1,26.9,3427.8,low_lubrication +4.1,65.2,66.8,23.3,2795.7,low_lubrication +7.3,66.7,49.7,16.2,1718.1,bearing_wear +7.2,76.0,84.1,31.5,3184.3,bearing_wear +1.8,95.2,40.6,19.5,2832.3,overheating +7.4,63.2,72.8,25.4,3349.5,low_lubrication +3.8,107.0,72.1,33.3,2491.9,overheating +5.7,77.5,65.1,25.1,2370.2,bearing_wear +4.5,95.9,65.2,28.7,2982.3,overheating +6.8,75.8,88.0,32.1,1441.0,bearing_wear +4.2,86.9,64.1,25.6,1828.7,overheating +4.1,101.6,59.3,24.5,2987.2,overheating +0.8,48.4,78.1,29.6,2477.5,normal +3.8,67.9,72.0,25.4,846.5,normal +4.6,68.6,44.7,19.5,2116.1,low_lubrication +9.0,77.6,47.3,15.7,3227.2,bearing_wear +3.3,93.3,32.0,16.1,3286.1,overheating +2.3,66.8,34.8,7.8,1264.3,normal +6.2,59.1,67.8,28.4,3140.5,low_lubrication +7.3,74.4,79.6,29.3,3198.2,bearing_wear +2.2,93.9,40.3,18.2,3097.9,overheating +6.0,69.8,63.4,22.5,2591.7,low_lubrication +2.4,98.7,74.8,34.2,2850.8,overheating +4.1,90.0,47.4,22.6,672.2,overheating +5.0,62.1,63.9,22.9,2168.3,low_lubrication +4.1,83.4,60.5,28.1,2733.0,overheating +7.1,71.6,75.4,27.5,1395.2,bearing_wear +2.8,68.8,60.3,22.4,1773.9,normal +3.5,81.1,41.9,22.4,2679.8,overheating +5.2,64.2,43.5,13.3,2347.0,low_lubrication +5.7,65.5,86.8,35.5,3970.4,low_lubrication +3.0,65.3,70.3,23.6,191.9,normal +3.9,54.4,44.2,14.9,2134.1,normal +1.7,93.5,37.0,17.4,1746.5,overheating +2.5,59.3,82.5,30.9,1171.0,normal +6.6,71.5,51.1,18.7,2462.7,bearing_wear +7.4,73.0,73.3,27.2,3203.5,bearing_wear +3.8,58.5,32.4,13.6,1313.2,normal +6.0,61.8,52.0,19.8,4030.3,low_lubrication +6.0,76.0,56.4,21.5,2416.7,bearing_wear +4.6,65.6,89.1,31.1,2948.4,low_lubrication +5.2,64.0,38.4,18.1,2519.0,low_lubrication +4.5,63.5,34.9,11.5,2794.9,low_lubrication +6.1,61.4,73.4,30.0,3687.1,low_lubrication +8.9,70.3,70.7,26.6,2213.0,bearing_wear +2.5,52.9,50.8,16.7,1544.2,normal +6.8,66.5,43.2,18.9,2853.9,low_lubrication +7.7,59.6,39.6,17.6,3239.7,bearing_wear +4.8,70.7,64.7,22.8,2202.0,low_lubrication +4.1,104.9,53.7,25.9,3468.8,overheating +4.4,74.4,44.4,17.7,2970.7,low_lubrication +1.1,83.0,50.3,23.5,591.5,overheating +1.4,59.7,74.1,24.6,1746.1,normal +10.7,77.3,87.1,28.5,2976.2,bearing_wear +5.7,53.8,81.4,31.6,2388.2,low_lubrication +8.5,72.2,71.8,27.0,2944.8,bearing_wear +2.0,99.3,39.3,20.4,770.6,overheating +4.9,62.6,34.3,11.5,3540.7,low_lubrication +4.1,74.6,73.4,29.3,4182.8,low_lubrication +1.2,87.1,45.8,20.1,1148.7,overheating +4.8,79.5,30.4,7.5,4187.1,low_lubrication +8.4,74.3,36.2,13.1,2896.4,bearing_wear +3.1,97.2,71.9,33.8,2486.6,overheating +8.7,75.6,32.7,9.8,3298.4,bearing_wear +5.0,91.2,43.3,20.5,885.9,overheating +5.8,67.2,88.0,33.2,2767.9,low_lubrication +4.8,66.3,88.0,31.8,2840.2,low_lubrication +3.7,71.2,67.9,21.7,2860.5,low_lubrication +5.5,70.3,87.0,31.9,2094.9,low_lubrication +5.5,68.8,59.3,21.6,2398.1,low_lubrication +2.1,61.5,35.8,13.4,1464.0,normal +1.8,90.8,56.0,23.9,596.7,overheating +4.2,96.4,78.8,32.6,1016.0,overheating +1.5,61.3,63.9,24.2,610.3,normal +2.6,53.6,36.5,12.3,2090.5,normal +5.9,57.8,37.0,12.2,2801.6,low_lubrication +5.0,99.6,73.3,32.9,2979.7,overheating +6.7,70.2,36.6,11.7,2797.1,bearing_wear +4.0,90.9,61.3,30.2,1505.5,overheating +1.0,76.3,85.0,29.2,1127.7,normal +7.7,71.7,36.1,14.7,2933.9,low_lubrication +3.2,94.7,31.6,17.0,2619.0,overheating +4.2,72.6,47.8,22.8,3476.6,overheating +5.5,65.6,88.7,34.6,3699.0,low_lubrication +7.4,68.5,59.8,24.9,2075.7,bearing_wear +4.2,65.6,58.2,14.7,2196.6,low_lubrication +3.3,93.4,51.5,22.3,1813.7,overheating +2.5,84.0,78.0,36.3,2352.5,overheating +3.1,94.9,43.5,21.6,2290.9,overheating +3.2,83.2,30.3,13.0,3101.3,overheating +5.8,57.6,85.7,29.2,3582.4,low_lubrication +6.9,71.2,56.5,20.8,2415.0,low_lubrication +4.9,92.2,68.5,31.1,3250.1,overheating +6.5,68.6,78.9,32.4,1453.4,bearing_wear +1.3,52.3,36.1,14.9,1257.9,normal +2.4,71.2,85.7,31.6,2204.0,normal +8.4,75.6,43.7,19.2,1568.6,bearing_wear +4.6,90.7,63.2,26.8,1192.0,overheating +3.0,56.0,39.3,10.3,1343.8,normal +4.5,101.7,59.1,26.8,1335.9,overheating +4.8,58.9,32.8,11.5,2003.5,low_lubrication +6.0,65.2,70.5,24.8,2561.7,low_lubrication +5.6,75.4,85.0,28.5,2422.8,bearing_wear +4.8,96.0,80.4,35.2,647.9,overheating +5.6,67.5,64.8,24.8,2964.1,low_lubrication +5.7,68.3,83.0,29.9,2033.8,bearing_wear +2.1,64.4,38.4,15.5,357.5,normal +3.0,91.2,58.3,25.7,2178.5,overheating +1.8,61.6,71.3,25.7,29.5,normal +2.4,75.6,75.0,31.5,2554.3,overheating +2.9,59.9,88.8,35.1,822.7,normal +1.0,55.0,60.1,21.2,364.6,normal +2.9,55.7,39.5,15.7,1231.2,normal +5.2,103.9,73.4,31.3,2418.6,overheating +3.2,50.5,79.6,26.6,2147.4,normal +7.8,83.7,30.4,12.3,1916.6,bearing_wear +5.3,64.1,59.1,21.9,2153.1,low_lubrication +5.1,63.4,49.3,21.3,3653.2,low_lubrication +8.3,82.3,55.3,20.8,1887.1,bearing_wear +2.8,83.2,37.7,18.0,2420.2,overheating +1.0,61.1,44.0,16.8,1438.2,normal +8.8,85.6,31.6,9.5,1363.6,bearing_wear +8.3,72.7,80.1,31.1,3201.1,bearing_wear +0.2,55.2,75.8,29.1,883.2,normal +2.5,64.9,86.2,32.4,459.5,normal +4.0,102.7,90.0,34.9,1231.3,overheating +6.3,82.1,89.8,31.3,2848.4,bearing_wear +7.5,69.0,45.4,18.9,2613.5,bearing_wear +11.0,70.7,48.7,16.7,1354.8,bearing_wear +5.2,71.3,81.1,30.2,2337.5,low_lubrication +1.2,46.7,85.1,31.8,2031.3,normal +5.5,68.8,60.7,22.4,1299.7,bearing_wear +3.0,89.7,70.2,30.6,3370.2,overheating +2.4,84.4,71.4,29.1,3121.0,overheating +3.7,66.1,72.3,28.6,619.9,normal +7.0,68.5,49.3,21.9,2057.9,bearing_wear +2.2,58.6,50.0,16.6,1449.2,normal +3.6,47.9,73.0,25.4,1726.2,normal +1.5,59.3,83.8,31.9,305.7,normal +5.5,63.6,44.7,11.0,3238.6,low_lubrication +3.7,79.0,39.2,17.2,2836.2,overheating +5.4,63.3,53.5,18.8,3274.6,low_lubrication +9.2,65.9,36.0,15.9,3186.3,bearing_wear +2.3,102.8,85.2,33.9,2018.0,overheating +5.8,62.9,63.9,26.0,3886.4,low_lubrication +5.4,70.2,89.0,29.1,2900.7,low_lubrication +3.8,86.8,50.9,29.6,2277.9,overheating +5.7,68.5,36.6,14.8,1656.4,bearing_wear +5.3,60.4,61.6,23.9,3317.5,low_lubrication +3.2,52.0,84.2,28.2,1953.7,normal +4.3,68.3,61.2,20.5,2065.4,bearing_wear +6.8,76.9,30.1,12.9,1263.4,bearing_wear +6.1,63.7,36.5,13.7,2527.7,low_lubrication +3.0,90.8,77.6,34.4,2166.9,overheating +5.1,84.5,85.9,33.8,731.0,overheating +4.2,89.2,84.8,35.3,2975.6,overheating +2.6,96.6,36.3,19.5,832.8,overheating +2.2,91.7,39.6,19.7,1581.5,overheating +3.0,65.4,66.9,25.9,251.3,normal +2.5,63.3,64.3,23.4,292.4,normal +4.3,62.2,50.9,21.2,2072.7,low_lubrication +5.2,70.2,54.1,21.8,2754.1,low_lubrication +1.6,51.3,47.7,17.8,2451.9,normal +7.8,79.2,60.4,20.7,3488.2,bearing_wear +1.4,61.7,80.3,30.5,1036.1,normal +3.9,62.0,78.3,29.7,202.0,normal +5.2,61.7,60.9,19.2,2278.9,low_lubrication +8.4,84.7,50.3,19.3,3278.9,bearing_wear +2.3,49.3,81.1,29.0,1392.3,normal +3.5,89.5,36.5,16.8,2322.0,overheating +4.4,64.9,87.5,29.5,3831.5,low_lubrication +4.4,64.9,57.8,22.7,3124.0,low_lubrication +2.8,63.5,41.6,15.2,1553.6,normal +8.6,65.4,41.2,12.4,3305.1,bearing_wear +3.4,86.4,71.3,32.1,2679.9,overheating +3.0,93.7,82.5,32.3,2259.2,overheating +10.2,77.1,54.2,20.4,1963.9,bearing_wear +4.2,56.0,38.6,15.8,2371.6,normal +9.1,71.6,69.5,26.0,1254.5,bearing_wear +7.5,74.6,41.7,16.6,2167.7,bearing_wear +1.7,62.2,52.2,15.6,1660.9,normal +1.8,60.4,81.6,30.2,1632.0,normal +5.5,68.3,68.3,22.4,3575.7,low_lubrication +4.2,52.5,79.5,26.6,637.8,normal +5.4,65.2,41.7,17.9,3329.8,low_lubrication +5.0,66.5,37.5,12.8,2508.0,low_lubrication +3.7,50.4,66.0,27.1,2329.6,normal +6.6,70.6,89.1,33.8,1682.1,bearing_wear +2.2,50.2,79.0,27.6,1456.7,normal +8.1,66.4,73.1,28.2,1331.2,bearing_wear +5.1,70.0,89.5,29.9,2998.0,low_lubrication +3.4,86.4,32.7,19.8,1605.6,overheating +4.0,65.8,37.4,15.0,2255.4,normal +2.8,61.7,36.9,11.3,414.7,normal +1.9,59.4,58.0,20.1,1975.6,normal +6.0,64.0,67.0,20.3,2601.8,bearing_wear +11.4,79.6,49.3,18.7,1559.7,bearing_wear +2.3,69.1,33.5,12.5,1707.5,normal +2.7,63.7,85.6,31.4,1092.8,normal +4.2,87.7,55.4,25.5,2412.2,overheating +3.8,92.7,63.9,28.2,1972.9,overheating +2.6,63.8,81.0,24.8,1936.0,normal +5.2,62.6,42.3,20.4,2931.2,low_lubrication +3.1,90.4,57.2,23.9,838.6,overheating +2.3,82.7,60.6,30.3,1234.2,overheating +3.2,85.4,80.8,30.3,1464.3,overheating +5.1,87.7,35.0,21.8,1606.0,overheating +4.9,60.2,86.0,30.8,3075.7,low_lubrication +10.1,72.7,49.8,16.7,3109.8,bearing_wear +4.1,54.4,56.0,15.7,1634.8,normal +3.3,86.7,82.6,38.1,1652.6,overheating +7.0,83.7,63.6,26.7,2717.1,bearing_wear +3.9,84.8,36.6,15.8,1677.6,overheating +5.5,63.5,39.3,17.2,3196.5,low_lubrication +4.0,72.3,48.7,17.3,3405.2,low_lubrication +10.5,75.6,84.1,28.4,1340.6,bearing_wear +2.1,66.2,48.1,18.3,550.0,normal +8.4,76.1,70.0,25.3,2592.1,bearing_wear +7.4,76.0,49.7,16.1,3422.9,bearing_wear +2.9,64.3,58.9,17.6,1242.6,normal +1.4,57.0,38.0,11.6,959.9,normal +3.1,71.4,76.3,26.5,841.0,normal +9.2,59.5,36.7,13.9,2463.0,bearing_wear +2.1,63.8,59.6,22.2,1179.5,normal +3.6,62.6,63.3,23.0,2232.8,normal +7.3,71.8,39.9,16.6,1579.3,bearing_wear +3.7,93.4,49.1,21.8,3370.7,overheating +2.8,57.7,71.9,25.0,1144.7,normal +6.0,71.8,81.2,27.8,2209.7,low_lubrication +5.5,74.1,38.0,16.4,1777.9,bearing_wear +2.3,60.6,75.3,28.1,2048.8,normal +3.0,99.4,89.4,42.4,823.4,overheating +8.1,65.8,75.9,26.9,1557.9,bearing_wear +3.9,83.2,32.6,19.1,3094.0,overheating +8.9,72.7,70.3,26.9,3073.7,bearing_wear +2.9,48.7,63.2,19.3,65.0,normal +8.8,71.6,87.9,32.8,3293.9,bearing_wear +4.8,74.9,51.7,16.6,3419.5,low_lubrication +6.1,75.7,32.1,16.0,2987.5,bearing_wear +3.3,69.1,33.3,13.0,80.4,normal +2.1,71.8,79.7,28.0,1428.9,normal +3.9,68.8,63.7,19.9,4094.1,low_lubrication +10.7,76.5,50.2,19.6,1534.8,bearing_wear +3.2,53.6,30.3,9.3,941.2,normal +6.2,71.0,83.4,31.1,1287.9,bearing_wear +3.6,57.7,49.0,18.0,2216.6,normal +3.0,96.3,72.4,34.6,2867.1,overheating +3.6,70.4,54.4,20.4,1331.0,normal +7.8,68.2,30.0,9.0,2544.6,bearing_wear +3.1,91.2,56.0,25.1,2274.4,overheating +1.9,64.7,53.2,20.5,1299.4,normal +4.4,72.3,68.3,26.2,2162.7,low_lubrication +3.5,97.2,81.8,33.0,1956.5,overheating +9.0,75.0,63.4,24.1,1649.1,bearing_wear +3.6,102.9,49.6,21.9,1153.5,overheating +2.1,60.4,46.3,12.7,2395.2,normal +6.6,62.1,53.4,21.2,3324.8,bearing_wear +3.3,93.4,81.3,38.5,2822.0,overheating +4.3,91.1,77.5,36.6,1541.2,overheating +1.2,56.8,37.0,14.7,804.0,normal +3.2,115.1,88.8,37.9,522.4,overheating +7.5,76.3,59.9,20.3,3180.8,bearing_wear +1.9,93.8,67.0,30.9,589.2,overheating +10.0,68.3,67.3,25.8,1555.4,bearing_wear +4.3,92.5,78.1,38.2,1130.0,overheating +4.6,59.4,42.7,20.2,2406.9,low_lubrication +5.0,69.0,47.9,19.5,3751.0,low_lubrication +5.9,70.0,44.3,14.0,2520.7,low_lubrication +3.9,62.9,46.7,14.4,2674.7,low_lubrication +1.8,56.0,78.2,29.0,1791.6,normal +0.7,59.0,76.8,25.1,1852.0,normal +2.6,93.2,46.1,19.2,1085.1,overheating +9.4,70.9,37.8,16.3,3492.3,bearing_wear +3.2,63.9,36.5,10.8,415.0,normal +6.3,62.4,59.6,20.3,3271.4,low_lubrication +3.9,97.8,45.4,20.2,1409.9,overheating +4.7,67.8,82.0,27.3,2592.9,bearing_wear +8.2,72.7,70.6,23.5,3218.1,bearing_wear +4.2,86.9,42.6,19.1,893.8,overheating +5.4,101.4,32.1,17.2,2208.7,overheating +9.7,77.2,67.9,20.6,1965.6,bearing_wear +5.6,70.0,33.1,10.3,3427.8,low_lubrication +6.6,79.7,78.5,27.2,3251.3,bearing_wear +2.7,91.0,41.0,23.2,1700.6,overheating +9.0,76.7,75.7,23.8,3003.1,bearing_wear +2.8,63.3,46.2,15.9,1698.7,normal +5.5,67.6,87.4,32.3,3398.0,low_lubrication +3.9,96.8,39.1,20.2,2757.4,overheating +1.7,54.0,65.3,23.6,62.6,normal +4.4,92.9,88.1,36.3,1616.5,overheating +2.3,43.3,85.5,29.8,1159.3,normal +1.6,55.9,31.4,12.4,1866.8,normal +1.5,61.4,89.7,33.0,1294.7,normal +2.6,59.0,54.5,18.9,2048.6,normal +8.8,80.2,52.4,18.2,1587.5,bearing_wear +6.8,76.4,58.8,22.2,3385.1,bearing_wear +3.1,59.9,79.5,26.2,2389.7,normal +7.3,66.6,81.9,26.9,3404.5,low_lubrication +4.0,63.8,39.6,11.6,3569.4,low_lubrication +1.6,55.9,60.3,20.0,1037.1,normal +6.3,66.7,83.2,33.2,2463.1,low_lubrication +3.6,90.7,30.8,17.3,1220.8,overheating +2.3,93.9,72.3,30.5,995.9,overheating +1.6,51.2,48.1,19.8,366.4,normal +2.9,61.1,76.6,28.0,856.5,normal +4.3,97.9,50.7,25.4,3463.7,overheating +2.8,57.7,35.7,11.4,677.8,normal +2.0,52.6,67.9,23.8,122.7,normal +3.1,104.9,56.6,24.5,2552.5,overheating +3.0,69.3,63.9,21.2,1521.6,normal +6.3,84.7,45.3,15.8,1605.0,bearing_wear +8.5,78.3,81.1,29.0,2782.0,bearing_wear +3.0,91.7,60.6,30.7,3453.5,overheating +6.3,75.3,46.7,17.6,2248.2,bearing_wear +2.4,56.9,59.4,20.4,165.9,normal +0.6,60.2,40.9,16.7,2187.7,normal +5.8,61.8,74.4,26.9,3000.2,low_lubrication +7.5,72.6,32.9,18.3,2431.5,bearing_wear +6.8,67.2,39.6,12.1,2741.8,bearing_wear +9.2,67.8,83.3,31.0,3103.0,bearing_wear +6.8,74.4,85.3,28.8,2042.3,bearing_wear +7.4,77.8,77.6,27.8,2634.1,bearing_wear +2.1,81.1,61.7,29.3,3146.3,overheating +9.9,74.4,49.9,15.9,1601.8,bearing_wear +9.0,67.6,30.2,7.8,1421.0,bearing_wear +4.5,102.8,72.1,34.4,2845.8,overheating +7.9,60.9,44.8,15.2,3756.0,low_lubrication +4.6,63.7,56.9,17.2,3143.4,low_lubrication +4.6,85.2,63.7,28.1,2480.5,overheating +8.6,80.8,30.1,8.9,1773.5,bearing_wear +3.3,90.6,32.8,20.8,706.9,overheating +2.0,105.8,60.0,26.1,2112.4,overheating +3.2,60.2,61.4,21.8,566.0,normal +3.3,59.0,37.9,13.1,1575.4,normal +7.7,73.8,89.1,32.1,1580.9,bearing_wear +3.0,60.8,51.9,13.2,369.9,normal +2.1,51.8,80.0,28.7,2477.2,normal +2.3,63.4,58.9,23.6,1509.9,normal +4.9,64.9,73.8,30.5,3436.0,low_lubrication +3.4,82.7,75.2,35.4,2827.4,overheating +2.4,54.9,55.0,20.3,624.2,normal +7.9,64.5,54.6,17.9,3007.7,bearing_wear +4.8,87.8,61.2,22.0,3487.0,bearing_wear +9.9,68.6,50.2,15.1,1563.5,bearing_wear +11.2,63.1,33.8,13.8,2334.4,bearing_wear +2.9,64.0,39.1,15.4,566.7,normal +2.9,60.8,65.3,24.6,1419.5,normal +6.3,62.8,64.4,21.9,3704.3,low_lubrication +3.9,56.7,69.9,26.7,1262.9,normal +2.1,59.4,55.1,20.1,1333.5,normal +5.9,62.5,39.6,13.6,2630.7,low_lubrication +2.6,82.8,44.9,23.3,2434.6,overheating +5.0,64.1,35.7,11.7,4057.8,low_lubrication +4.2,66.5,65.4,23.9,886.8,normal +5.0,59.5,55.1,18.9,4011.7,low_lubrication +2.5,66.5,77.0,25.6,554.6,normal +7.3,72.2,88.0,29.1,2736.5,bearing_wear +4.8,63.6,83.3,29.9,3593.6,low_lubrication +5.4,62.0,46.9,16.9,3298.3,low_lubrication +6.7,78.2,63.9,24.9,1221.5,bearing_wear +4.7,66.9,70.6,22.4,2741.4,low_lubrication +10.0,69.2,35.9,11.4,1949.8,bearing_wear +1.0,66.0,40.0,15.7,1156.5,normal +3.6,62.5,30.9,13.0,1045.7,normal +7.7,60.1,80.8,27.4,2617.5,bearing_wear +5.8,55.4,51.8,16.0,2778.0,low_lubrication +5.5,71.2,45.1,22.3,3734.7,low_lubrication +2.8,57.1,31.5,11.8,1861.4,normal +4.9,90.0,69.4,25.9,2860.9,overheating +2.8,89.2,77.2,34.7,2586.1,overheating +2.8,63.6,63.4,27.3,1434.6,normal +4.3,68.5,60.7,23.7,3140.2,low_lubrication +4.2,62.6,33.4,12.9,3997.5,low_lubrication +3.4,63.0,64.7,21.7,2453.2,normal +5.1,64.2,33.0,14.2,2460.3,low_lubrication +3.4,65.4,67.8,22.4,3894.4,low_lubrication +6.4,79.6,34.8,12.7,2968.1,bearing_wear +2.7,63.1,88.0,32.9,564.4,normal +2.3,54.2,35.1,10.5,734.9,normal +8.4,72.7,64.0,19.1,2735.1,bearing_wear +1.8,56.0,56.9,22.8,320.5,normal +4.0,99.6,86.8,38.0,2892.0,overheating +4.1,73.0,77.3,29.4,4188.9,low_lubrication +1.5,62.0,64.9,21.7,2057.3,normal +5.5,63.6,56.1,21.1,4138.9,low_lubrication +10.4,69.3,65.4,23.1,2265.0,bearing_wear +5.4,60.0,86.7,34.9,3318.8,low_lubrication +4.7,97.8,57.6,25.0,1396.9,overheating +5.9,79.5,77.7,26.0,3067.1,bearing_wear +3.4,94.5,58.0,26.1,2287.9,overheating +8.2,71.3,61.4,20.7,2263.2,bearing_wear +3.4,63.6,53.9,20.0,1823.5,normal +4.8,87.4,56.9,26.4,2103.0,overheating +3.4,88.7,72.3,32.3,1439.7,overheating +2.0,52.1,75.8,26.9,1372.4,normal +2.2,88.1,83.5,35.4,734.5,overheating +5.5,72.9,39.2,15.8,3038.2,bearing_wear +4.9,65.4,72.9,29.2,3351.0,low_lubrication +2.2,100.6,80.3,39.2,2571.2,overheating +3.0,103.7,60.6,28.2,1777.0,overheating +1.8,44.1,58.5,21.4,554.3,normal +7.4,68.9,67.7,23.0,3284.5,bearing_wear +4.4,68.9,53.5,18.6,3186.3,low_lubrication +3.6,104.0,58.3,25.6,924.1,overheating +7.9,76.7,65.7,24.2,2777.9,bearing_wear +2.3,90.9,56.0,26.4,1485.3,overheating +9.0,67.5,62.0,21.0,1430.8,bearing_wear +8.5,66.2,36.1,15.0,3164.4,bearing_wear +8.9,73.2,53.3,20.2,2836.2,bearing_wear +6.7,65.2,71.7,27.8,3088.7,low_lubrication +3.0,67.4,40.2,14.8,656.0,normal +2.1,57.3,35.4,13.9,114.2,normal +3.2,59.9,84.2,26.3,888.2,normal +7.7,77.8,63.2,18.7,3288.3,bearing_wear +2.3,60.4,39.0,14.4,1790.3,normal +5.1,56.9,40.7,13.2,3005.6,low_lubrication +7.5,67.2,78.1,25.9,2150.6,bearing_wear +3.3,78.9,68.3,29.4,2287.9,overheating +2.7,56.8,63.6,19.9,2308.3,normal +2.4,74.3,49.0,12.9,1761.0,normal +3.1,47.0,76.7,30.5,1598.0,normal +5.7,72.7,31.5,15.7,3984.3,low_lubrication +3.1,62.3,59.2,23.4,249.9,normal +4.3,68.3,59.6,23.6,2437.8,low_lubrication +5.8,61.2,41.2,15.4,2182.2,low_lubrication +5.2,71.0,78.6,34.7,3125.3,low_lubrication +4.8,109.6,80.1,32.9,836.6,overheating +6.1,76.0,87.0,32.6,2114.6,low_lubrication +6.7,70.7,38.6,14.9,2460.5,bearing_wear +1.5,63.3,48.6,13.7,1400.9,normal +6.0,71.8,54.0,17.5,3376.1,low_lubrication +4.3,102.1,39.7,21.4,744.8,overheating +8.5,79.5,54.1,20.7,3128.9,bearing_wear +3.5,88.0,58.1,31.9,2403.0,overheating +5.0,69.4,50.7,18.0,2602.4,low_lubrication +6.0,57.4,43.0,17.8,3731.9,low_lubrication +2.3,97.0,39.6,23.8,1659.7,overheating +1.9,58.7,44.0,16.4,248.8,normal +2.9,90.1,72.2,28.3,3228.4,overheating +3.2,87.0,36.8,19.3,1317.7,overheating +5.6,71.1,42.4,13.6,3743.2,low_lubrication +2.0,96.5,50.2,27.3,2537.7,overheating +7.7,65.4,35.7,11.0,3227.6,bearing_wear +1.9,82.2,49.7,23.7,3065.2,overheating +5.9,83.6,60.4,22.4,1890.6,bearing_wear +6.2,72.6,61.7,21.7,2196.8,low_lubrication +1.7,53.1,85.0,29.0,466.7,normal +3.0,55.6,69.7,26.0,693.0,normal +8.2,84.9,79.0,29.9,2105.7,bearing_wear +4.0,62.5,54.1,20.4,4189.8,low_lubrication +0.3,63.4,77.2,27.3,2360.4,normal +5.3,96.8,48.2,23.0,1924.2,overheating +6.0,62.0,76.9,25.2,3708.7,low_lubrication +6.5,65.0,43.6,15.7,4167.9,low_lubrication +3.8,68.5,43.6,12.3,3476.3,low_lubrication +4.4,75.6,64.4,22.5,3801.6,low_lubrication +2.7,87.2,40.7,21.5,2633.3,overheating +4.1,96.7,56.9,28.7,2519.5,overheating +2.2,66.7,48.6,18.1,1277.9,normal +5.2,81.3,46.0,22.2,3008.7,overheating +3.8,58.3,85.3,26.7,1743.2,normal +2.3,88.6,39.4,22.3,567.7,overheating +3.8,61.5,50.9,17.1,2972.6,low_lubrication +5.4,65.6,52.8,18.4,2736.2,low_lubrication +6.3,72.7,80.3,31.0,2086.8,bearing_wear +6.8,65.5,57.9,20.9,2628.6,low_lubrication +1.5,60.2,44.0,16.8,1168.2,normal +4.5,75.0,72.3,28.4,2236.6,low_lubrication +2.2,55.2,38.4,14.0,958.4,normal +2.6,91.1,74.1,31.7,3102.6,overheating +3.3,51.7,83.8,31.6,1967.4,normal +5.0,71.1,38.3,15.3,2222.7,low_lubrication +6.9,70.3,65.5,26.5,2731.6,bearing_wear +6.0,62.6,33.5,13.4,3817.6,low_lubrication +6.5,61.9,35.2,11.7,1495.1,bearing_wear +3.8,55.7,86.2,31.1,1113.2,normal +2.4,49.9,85.9,32.6,1347.9,normal +10.5,71.1,37.0,15.1,1462.6,bearing_wear +2.6,61.7,88.3,34.9,721.4,normal +6.6,67.8,31.6,10.5,2672.8,bearing_wear +5.3,62.4,75.2,27.3,3476.3,low_lubrication +1.6,63.7,59.6,22.7,1609.4,normal +9.0,70.1,30.4,7.5,2269.2,bearing_wear +5.0,88.0,31.4,19.8,1141.8,overheating +4.0,69.6,63.3,21.1,3301.6,low_lubrication +2.6,56.2,80.2,29.8,1667.1,normal +6.7,70.4,39.7,13.5,3586.7,low_lubrication +6.5,72.3,58.2,22.7,2490.0,low_lubrication +8.8,78.4,69.4,21.2,3221.7,bearing_wear +4.1,89.6,60.3,27.8,2668.1,overheating +3.3,92.8,59.2,27.3,1733.4,overheating +2.9,54.7,74.7,26.9,253.0,normal +3.1,55.4,74.1,23.7,90.6,normal +5.8,64.2,33.6,13.5,2751.5,low_lubrication +5.5,75.2,41.1,10.2,3056.7,low_lubrication +7.3,64.7,83.2,34.4,3042.3,low_lubrication +5.1,60.2,35.8,13.0,3112.2,low_lubrication +6.0,65.4,43.9,17.7,4178.0,low_lubrication +3.1,49.1,70.2,26.6,2068.6,normal +7.1,75.2,49.3,20.5,1846.3,bearing_wear +1.5,60.7,57.8,18.5,1387.6,normal +5.0,59.1,70.2,27.3,4009.6,low_lubrication +8.3,72.8,73.9,26.7,1432.6,bearing_wear +3.9,89.1,47.2,21.9,1603.8,overheating +2.7,95.9,79.1,32.0,2566.4,overheating +2.6,53.9,39.8,15.3,642.9,normal +4.2,93.8,66.0,28.5,2124.5,overheating +2.5,58.1,81.0,31.3,1895.1,normal +3.5,97.9,53.3,23.0,2148.4,overheating +6.9,63.2,36.8,15.0,2749.1,bearing_wear +6.2,70.1,40.3,14.3,3278.7,low_lubrication +4.6,96.6,49.8,25.0,2882.1,overheating +3.3,93.5,88.6,40.9,2713.4,overheating +2.0,61.7,57.5,18.9,1371.0,normal +1.5,50.4,77.2,25.6,18.7,normal +3.1,57.5,83.6,29.5,1665.1,normal +5.3,64.4,51.9,19.7,2439.9,low_lubrication +4.8,91.9,85.4,37.7,3256.6,overheating +5.0,65.6,56.8,21.4,3343.8,low_lubrication +5.9,74.7,67.4,22.9,2247.6,bearing_wear +1.7,61.4,61.2,21.9,1363.0,normal +7.7,63.6,32.4,8.7,3122.0,bearing_wear +6.3,64.0,45.2,19.1,2762.4,low_lubrication +4.1,71.5,89.5,32.7,79.0,normal +3.4,88.3,80.9,37.9,3388.4,overheating +3.2,94.9,30.7,19.7,2331.5,overheating +3.9,91.6,66.4,32.6,1720.5,overheating +5.3,56.8,36.5,16.7,2577.4,low_lubrication +2.9,84.1,60.4,28.3,1936.2,overheating +2.8,96.1,67.6,29.2,2086.4,overheating +2.9,90.3,80.3,36.5,1328.9,overheating +8.7,68.1,49.7,20.3,2638.6,bearing_wear +8.1,70.8,87.8,31.6,1952.4,bearing_wear +8.9,73.4,45.1,16.1,3006.3,bearing_wear +8.1,69.7,53.5,20.3,2793.3,bearing_wear +3.0,96.2,47.0,27.2,1958.5,overheating +2.6,58.6,77.7,26.4,2218.6,normal +3.3,93.4,81.9,34.7,998.2,overheating +2.4,91.0,42.1,18.2,3139.2,overheating +3.4,63.5,59.3,20.3,1661.4,normal +7.5,78.1,83.9,30.4,1598.6,bearing_wear +7.5,68.8,89.3,32.9,2189.5,bearing_wear +3.3,59.4,53.6,21.6,969.7,normal +3.0,88.6,36.9,19.4,3201.9,overheating +3.8,86.0,44.7,21.1,2877.9,overheating +4.2,67.1,72.0,23.9,3551.6,low_lubrication +5.4,67.5,66.6,26.2,2709.6,low_lubrication +5.2,54.3,86.9,33.5,4150.9,low_lubrication +6.8,68.8,89.9,33.4,2311.5,bearing_wear +6.8,66.6,42.9,15.0,1593.8,bearing_wear +4.8,69.9,46.2,18.1,2365.7,bearing_wear +4.2,93.2,73.8,32.9,908.6,overheating +3.4,90.1,88.1,42.3,2683.9,overheating +5.6,67.5,89.2,29.3,2661.4,low_lubrication +4.6,69.6,35.8,15.1,3748.6,low_lubrication +2.6,62.3,54.7,20.4,2419.8,normal +5.5,72.0,46.8,15.7,3828.4,low_lubrication +5.8,65.1,38.2,11.1,2700.4,low_lubrication +9.4,70.8,56.5,20.5,2917.8,bearing_wear +2.5,89.2,34.3,19.8,1581.9,overheating +0.9,50.6,48.9,17.5,2495.6,normal +2.0,89.8,52.4,23.2,3384.6,overheating +3.6,73.8,60.3,26.3,669.6,overheating +4.3,89.3,88.3,37.4,1294.1,overheating +5.0,60.1,87.2,30.4,2238.3,low_lubrication +6.5,75.7,56.8,19.4,3298.9,bearing_wear +5.6,59.0,72.1,26.7,3026.3,low_lubrication +8.2,63.3,49.8,19.0,1298.3,bearing_wear +9.2,75.9,80.6,33.1,3156.6,bearing_wear +2.7,55.0,35.5,10.6,233.9,normal +6.6,72.3,34.6,9.7,2038.2,low_lubrication +6.8,79.7,82.2,30.6,1706.2,bearing_wear +2.3,98.5,82.3,33.2,3090.3,overheating +2.9,93.0,65.9,29.4,1079.1,overheating +5.2,64.5,63.6,21.5,4110.8,low_lubrication +1.1,88.2,62.3,26.2,1090.7,overheating +3.7,86.8,31.5,18.2,852.3,overheating +5.9,70.1,38.7,15.0,3904.4,low_lubrication +5.9,66.1,87.7,29.7,3857.2,low_lubrication +5.2,99.4,36.6,20.4,534.3,overheating +2.7,86.5,41.7,18.9,3216.8,overheating +1.8,83.8,67.5,30.4,1846.2,overheating +5.1,68.1,43.7,16.5,2477.6,low_lubrication +5.9,78.3,58.8,20.6,2111.1,bearing_wear +2.8,62.2,72.0,28.8,2072.7,normal +5.8,76.2,57.7,21.4,2147.8,bearing_wear +2.8,60.1,37.1,9.7,377.5,normal +7.6,67.3,43.5,16.1,2042.2,bearing_wear +8.7,72.3,74.7,27.1,3451.6,bearing_wear +8.5,82.0,56.8,20.5,1826.9,bearing_wear +5.1,78.9,76.4,26.9,3086.0,bearing_wear +9.0,64.2,47.1,13.3,2051.1,bearing_wear +5.6,64.7,35.1,12.4,2730.9,low_lubrication +3.1,65.5,76.3,26.2,2048.1,normal +1.7,95.8,51.4,21.6,595.8,overheating +5.1,73.4,69.9,24.5,1357.7,bearing_wear +4.4,63.0,68.0,23.4,1993.8,normal +5.1,85.3,78.7,33.1,2558.0,overheating +2.8,68.2,70.3,23.9,2116.5,normal +5.1,72.5,70.6,24.9,2711.5,bearing_wear +3.1,95.2,44.8,21.9,595.0,overheating +4.0,93.5,34.0,19.3,3123.7,overheating +3.5,89.3,31.0,17.7,1366.6,overheating +3.8,93.8,73.3,31.9,920.5,overheating +2.0,63.1,70.9,26.9,125.9,normal +8.1,70.1,78.0,26.5,1468.6,bearing_wear +3.0,95.2,60.6,28.1,1726.0,overheating +2.4,64.5,49.8,17.7,2375.0,normal +11.7,61.1,49.0,18.7,1655.8,bearing_wear +2.4,68.0,69.2,23.5,2093.5,normal +8.4,70.7,81.6,29.2,1422.2,bearing_wear +4.3,60.7,66.3,22.6,2476.7,low_lubrication +1.1,61.5,34.3,16.9,214.7,normal +7.1,71.8,48.4,19.3,1477.1,bearing_wear +2.9,55.8,37.2,15.7,1050.2,normal +2.8,98.6,46.8,19.6,2290.7,overheating +4.8,63.7,73.6,25.0,2117.2,low_lubrication +6.1,68.3,71.9,26.0,3885.4,low_lubrication +4.6,67.9,46.5,16.1,4026.4,low_lubrication +5.5,62.6,72.1,29.6,3963.3,low_lubrication +3.6,79.7,69.5,29.3,2877.4,overheating +3.8,99.5,64.8,29.8,2526.7,overheating +2.8,94.9,31.9,17.1,1638.2,overheating +7.2,80.7,66.5,26.2,2220.4,bearing_wear +6.6,70.2,68.3,24.0,1941.8,bearing_wear +6.8,56.5,86.1,32.4,2103.3,bearing_wear +2.5,69.5,32.5,12.9,1003.3,normal +5.3,72.5,31.4,12.0,3472.2,low_lubrication +2.3,97.3,42.5,21.7,2444.4,overheating +5.9,60.1,55.0,21.8,3238.3,low_lubrication +8.4,70.3,85.1,31.9,2167.3,bearing_wear +3.3,73.9,49.0,14.0,2283.3,normal +4.3,96.4,30.2,16.5,3484.8,overheating +8.3,72.8,54.4,20.3,2008.6,bearing_wear +1.6,84.7,48.7,25.6,3476.4,overheating +2.9,90.4,40.4,21.7,3188.8,overheating +5.6,68.5,51.3,16.8,3137.7,low_lubrication +3.9,57.2,79.6,33.8,811.0,normal +1.8,65.1,55.9,19.0,938.2,normal +7.7,64.5,33.7,11.3,3427.9,bearing_wear +3.8,65.8,56.3,17.9,1549.8,normal +8.0,77.8,35.8,11.6,2187.0,bearing_wear +3.6,80.2,50.9,21.6,1509.4,overheating +2.8,62.5,81.5,32.3,1893.4,normal +8.2,74.9,86.3,30.2,2417.1,bearing_wear +7.4,74.2,39.7,10.3,1432.8,bearing_wear +3.1,93.5,62.1,22.5,3429.6,overheating +7.0,63.5,48.3,17.9,2541.1,low_lubrication +2.2,79.5,44.4,23.1,1342.9,overheating +5.0,72.4,56.3,18.0,2256.3,low_lubrication +1.6,62.6,52.3,16.4,1631.4,normal +3.8,85.1,30.1,18.5,2693.6,overheating +6.6,70.8,67.9,26.6,2949.1,low_lubrication +2.4,53.1,73.1,27.6,99.9,normal +4.8,62.1,39.1,10.7,3914.5,low_lubrication +5.7,65.9,34.9,9.8,4163.2,low_lubrication +3.7,65.0,39.2,13.5,2399.0,normal +5.5,73.0,45.8,16.8,2227.4,low_lubrication +4.3,62.6,65.2,26.3,2616.8,low_lubrication +3.7,65.9,60.8,19.3,1710.0,normal +3.5,94.7,63.5,31.3,1677.1,overheating +1.8,98.7,82.1,37.0,1158.4,overheating +10.9,79.7,40.4,13.2,2849.0,bearing_wear +3.0,67.0,54.2,18.0,510.0,normal +4.6,90.0,85.0,37.4,1122.0,overheating +3.4,91.4,62.1,28.6,1241.1,overheating +6.6,73.9,43.2,16.1,2093.0,bearing_wear +5.3,65.2,65.8,24.0,4079.9,low_lubrication +1.8,51.0,86.4,32.1,2466.7,normal +2.8,85.9,45.8,25.3,1088.4,overheating +5.6,64.9,61.5,24.3,2865.0,bearing_wear +5.7,72.4,68.5,23.6,3471.4,low_lubrication +10.5,67.9,41.6,13.0,1559.2,bearing_wear +6.6,81.6,33.3,12.1,3087.8,bearing_wear +4.3,88.1,54.2,21.1,2496.6,overheating +7.5,75.2,52.8,19.3,2005.8,bearing_wear +2.0,51.4,33.1,13.0,30.4,normal +3.4,49.4,68.2,24.5,60.6,normal +6.8,68.5,37.7,11.3,3222.2,bearing_wear +9.4,75.8,41.3,15.8,3407.6,bearing_wear +9.7,70.4,66.6,21.3,2471.1,bearing_wear +5.3,64.4,38.2,14.2,2557.1,low_lubrication +5.4,71.4,59.9,19.0,3100.6,low_lubrication +7.3,64.6,72.5,22.8,2281.0,bearing_wear +4.0,76.6,89.1,38.5,1679.0,overheating +5.8,64.8,79.2,24.0,2321.4,low_lubrication +2.2,61.9,39.3,12.9,2326.8,normal +3.0,82.4,87.4,38.0,3331.1,overheating +3.2,86.3,46.4,20.4,1002.2,overheating +2.4,61.5,36.4,12.9,986.1,normal +3.3,57.0,83.4,27.6,581.4,normal +5.7,67.2,47.4,16.0,2182.9,low_lubrication +7.9,75.4,48.1,16.4,2005.5,bearing_wear +6.5,65.3,75.7,25.4,1223.3,bearing_wear +3.4,56.2,39.2,17.0,2333.9,low_lubrication +5.6,73.4,59.8,24.2,2047.3,bearing_wear +4.1,66.3,35.3,13.8,2393.2,low_lubrication +3.3,59.0,53.3,19.7,2285.1,normal +4.2,96.3,34.5,16.8,701.0,overheating +6.8,61.4,37.2,13.7,2046.9,low_lubrication +2.5,57.6,49.1,23.4,2030.7,normal +2.9,85.6,57.8,23.4,3346.5,overheating +4.6,55.9,72.4,26.6,2431.2,low_lubrication +0.7,99.9,84.5,34.4,2469.7,overheating +5.8,100.9,37.4,16.7,1670.5,overheating +6.1,69.4,88.9,31.8,2965.5,low_lubrication +3.2,69.2,52.9,20.8,408.0,normal +3.7,99.6,41.7,20.7,2583.3,overheating +6.0,75.2,32.7,11.6,2208.2,low_lubrication +4.7,64.3,67.8,25.0,2414.4,low_lubrication +3.2,96.0,80.3,37.0,1046.9,overheating +3.1,90.5,50.5,23.8,2355.4,overheating +5.4,73.7,60.4,21.2,2308.4,low_lubrication +2.5,60.6,31.4,9.1,1049.4,normal +12.4,82.2,51.3,21.9,1506.5,bearing_wear +3.6,70.8,30.3,11.4,3135.2,low_lubrication +4.8,60.0,78.1,29.0,2852.0,low_lubrication +6.1,78.5,48.0,15.4,2537.2,bearing_wear +8.8,64.1,42.2,13.4,2779.5,bearing_wear +3.3,100.2,42.5,24.6,942.1,overheating +6.7,62.4,67.3,26.3,3270.3,low_lubrication +2.6,87.9,42.3,19.6,2969.1,overheating +4.1,68.6,30.6,8.3,2006.7,bearing_wear +1.6,96.9,46.4,22.6,2956.5,overheating +4.9,66.9,50.2,18.1,2152.4,low_lubrication +7.4,73.0,47.3,19.5,1613.5,bearing_wear +2.1,54.5,56.9,18.7,815.4,normal +5.1,67.1,59.7,24.8,3090.2,low_lubrication +2.6,63.9,69.0,26.7,1525.7,normal +2.1,73.4,36.5,13.1,2237.9,normal +7.7,70.2,34.3,14.6,2433.5,bearing_wear +2.2,66.3,48.2,14.7,2209.2,normal +0.2,63.6,56.8,20.9,537.8,normal +5.3,72.4,62.5,22.6,2266.7,low_lubrication +4.3,65.4,63.4,23.7,4101.9,low_lubrication +0.4,68.9,68.9,24.4,2054.7,normal +4.9,99.3,54.7,28.2,1918.9,overheating +3.3,54.4,37.2,14.0,2337.4,normal +4.9,68.6,61.8,23.5,3544.3,low_lubrication +2.8,72.1,78.0,26.5,541.6,normal +5.9,75.3,60.2,19.6,3222.2,bearing_wear +2.4,55.8,64.3,24.7,387.5,normal +2.2,57.3,73.3,28.5,241.9,normal +2.0,59.3,60.5,18.6,2436.3,normal +4.0,91.3,45.5,21.0,2375.3,overheating +6.3,71.1,84.2,26.6,2526.2,low_lubrication +8.2,79.3,51.1,20.8,2558.8,bearing_wear +5.3,69.0,87.0,31.5,4121.9,low_lubrication +10.0,68.8,75.8,31.1,2617.0,bearing_wear +2.8,68.1,33.4,11.4,2348.9,normal +7.7,68.5,39.1,15.4,1786.3,bearing_wear +1.7,59.3,45.4,20.3,2376.7,normal +10.7,60.1,82.0,34.1,1600.2,bearing_wear +2.4,88.6,53.4,23.0,668.4,overheating +4.8,82.7,38.2,19.3,2543.2,overheating +6.5,58.7,56.8,20.0,2769.3,low_lubrication +3.9,67.0,80.7,30.5,3867.5,low_lubrication +3.4,53.4,47.8,18.8,1956.9,normal +2.6,88.2,68.7,33.3,2143.4,overheating +9.7,67.5,81.6,30.2,2685.8,bearing_wear +5.9,64.0,38.8,10.9,3357.8,low_lubrication +3.5,89.1,61.7,29.6,2213.3,overheating +3.8,60.3,57.7,25.4,2280.6,low_lubrication +5.4,74.0,31.3,12.6,2049.1,normal +4.6,61.0,53.8,18.2,3339.5,low_lubrication +11.1,73.5,56.8,22.9,2120.1,bearing_wear +6.6,56.9,64.4,22.7,2997.4,low_lubrication +4.6,56.4,87.7,33.0,2283.6,normal +8.6,66.5,49.9,18.3,2151.6,bearing_wear +2.0,56.8,46.8,17.8,2466.6,normal +9.0,74.7,43.9,15.8,2381.6,bearing_wear +7.8,76.3,35.2,14.1,3216.3,bearing_wear +6.1,66.7,59.8,20.6,2829.1,low_lubrication +6.1,82.3,55.9,21.2,2987.5,bearing_wear +6.5,71.2,44.7,13.7,2335.9,low_lubrication +5.7,64.6,60.5,21.8,2498.3,low_lubrication +0.9,48.8,37.4,13.6,727.6,normal +5.2,84.7,78.3,30.3,2093.0,overheating +3.5,56.6,58.4,23.1,1394.0,normal +2.9,102.8,35.8,18.7,1280.0,overheating +4.9,89.0,85.2,34.9,2348.2,overheating +7.0,73.9,46.5,17.2,2415.2,bearing_wear +0.5,45.9,67.0,24.4,686.4,normal +1.5,60.8,71.7,26.2,1312.7,normal +10.2,56.1,54.8,18.2,1539.8,bearing_wear +5.6,80.7,48.1,23.3,1507.5,bearing_wear +2.0,73.4,86.0,34.5,991.4,normal +4.3,59.1,75.4,27.6,2681.1,low_lubrication +7.4,75.0,72.3,23.7,1895.4,bearing_wear +4.9,62.0,88.4,31.9,2612.7,low_lubrication +2.4,102.4,39.6,17.2,1640.0,overheating +3.5,56.5,80.2,28.3,801.1,normal +1.0,60.5,37.5,17.1,260.6,normal +2.7,58.4,35.3,12.5,2089.7,normal +2.9,59.1,76.8,26.2,414.4,normal +0.8,70.7,40.9,13.5,426.0,normal +6.6,63.7,39.8,11.6,2399.2,low_lubrication +3.2,55.8,47.0,14.9,2066.0,normal +3.8,70.2,64.6,22.6,3575.7,low_lubrication +4.9,64.1,36.6,14.7,2320.5,low_lubrication +8.2,70.4,75.7,26.8,2756.5,bearing_wear +7.9,69.3,81.6,27.4,3000.5,bearing_wear +6.0,84.2,30.8,18.2,3004.5,overheating +5.1,63.3,59.6,23.2,2550.6,low_lubrication +6.3,60.8,69.3,22.1,4197.7,low_lubrication +6.0,67.4,63.3,21.8,2608.9,low_lubrication +5.6,87.5,36.9,22.3,1662.3,overheating +2.2,89.5,41.1,21.5,2286.6,overheating +5.8,100.3,56.0,24.1,727.7,overheating +5.1,67.6,56.4,23.1,2031.9,bearing_wear +7.4,69.6,68.1,23.3,1200.7,bearing_wear +5.6,76.2,42.3,15.2,3395.5,bearing_wear +6.2,83.2,83.6,28.1,2761.7,bearing_wear +1.8,96.3,76.4,33.4,1584.8,overheating +6.5,66.3,68.0,25.3,3999.2,low_lubrication +5.8,84.3,63.8,22.5,1589.4,bearing_wear +5.5,61.5,89.0,33.2,2704.3,low_lubrication +4.4,74.0,45.8,16.0,4121.4,low_lubrication +2.1,51.8,58.7,20.2,1324.1,normal +3.2,104.4,58.9,27.2,606.0,overheating +6.6,71.2,71.7,26.6,2512.1,bearing_wear +10.1,71.4,73.3,25.7,3459.1,bearing_wear +4.2,99.2,33.0,17.3,2207.9,overheating +6.1,64.7,49.3,18.3,2277.7,low_lubrication +5.6,77.5,65.2,23.8,3461.6,low_lubrication +9.6,82.7,88.4,28.8,2707.4,bearing_wear +2.6,77.1,43.9,21.7,1214.1,overheating +2.2,65.5,60.9,20.2,2356.3,normal +7.4,77.4,34.3,7.1,2079.6,bearing_wear +5.9,67.2,67.5,23.6,3680.1,low_lubrication +4.9,92.0,78.2,33.3,569.7,overheating +2.6,46.4,88.8,31.4,1025.9,normal +2.4,95.0,37.9,17.3,2897.5,overheating +3.9,65.1,79.0,25.6,4053.1,low_lubrication +3.5,82.2,40.8,18.1,2669.9,overheating +5.6,66.0,52.4,22.9,3443.5,low_lubrication +2.8,60.3,46.2,15.1,1144.3,normal +6.9,57.4,45.4,18.3,4185.3,low_lubrication +5.1,72.0,78.1,31.1,2562.8,low_lubrication +4.5,89.1,82.0,36.3,917.9,overheating +5.4,65.2,62.9,24.7,2408.4,bearing_wear +7.0,82.6,72.6,27.8,1656.1,bearing_wear +7.7,87.1,40.9,15.4,2560.4,bearing_wear +8.1,75.2,81.0,28.5,1270.5,bearing_wear +3.5,60.2,56.5,21.3,1163.0,normal +6.0,65.0,81.6,32.9,3170.7,low_lubrication +0.9,89.9,52.3,25.8,709.4,overheating +2.8,91.4,86.6,36.4,1492.4,overheating +2.7,62.7,60.9,22.7,6.8,normal +4.2,84.0,49.9,23.1,2299.9,overheating +4.4,64.7,83.1,32.0,4197.9,low_lubrication +9.3,64.0,30.2,11.3,2199.6,bearing_wear +1.7,88.7,55.5,29.1,1167.6,overheating +3.8,69.5,84.3,29.3,3951.5,low_lubrication +3.4,66.5,72.6,26.7,2.5,normal +2.8,94.9,78.7,37.8,3080.5,overheating +2.9,93.3,39.2,22.9,2665.7,overheating +7.4,62.6,48.2,14.4,1946.0,bearing_wear +6.7,66.4,79.5,26.5,3788.4,low_lubrication +3.9,91.8,80.7,34.7,2097.5,overheating +3.5,89.2,64.5,27.2,1238.2,overheating +7.1,74.7,43.3,16.3,2187.7,bearing_wear +3.5,101.6,88.0,38.7,3225.9,overheating +4.6,90.1,60.8,26.7,2668.5,overheating +3.4,58.5,38.7,15.2,713.1,normal +5.3,61.7,30.8,14.2,3670.5,low_lubrication +6.5,66.2,81.4,28.5,1815.3,bearing_wear +4.7,63.1,81.8,29.2,2720.3,low_lubrication +2.7,98.3,72.0,32.4,1085.2,overheating +2.9,63.3,37.9,14.6,2297.8,normal +4.1,79.1,40.3,11.8,2318.6,low_lubrication +6.0,75.1,65.9,23.1,1221.7,bearing_wear +2.2,104.5,41.4,20.8,2907.5,overheating +8.3,72.4,83.1,30.2,1864.2,bearing_wear +5.9,68.3,88.6,35.2,2805.3,low_lubrication +4.2,64.1,71.3,23.9,2007.4,low_lubrication +3.0,92.3,42.3,24.3,2843.3,overheating +3.0,70.9,47.1,20.6,643.3,overheating +8.5,69.7,85.4,30.6,2063.8,bearing_wear +9.9,76.1,36.0,14.6,3278.7,bearing_wear +8.7,57.8,71.4,26.3,2575.3,bearing_wear +6.5,80.9,60.9,18.2,1473.4,bearing_wear +2.9,69.7,84.5,34.0,2256.2,normal +5.5,70.1,42.6,13.5,2857.6,low_lubrication +2.7,56.8,48.8,16.7,421.6,normal +2.6,60.8,40.0,15.1,1357.4,normal +4.8,86.8,87.7,38.1,912.2,overheating +2.5,67.5,60.4,22.1,1161.0,normal +2.1,87.7,45.3,22.2,3410.3,overheating +1.6,65.3,52.2,14.9,2182.5,normal +1.4,89.2,64.5,31.3,2857.8,overheating +6.2,65.0,48.2,15.2,3886.5,low_lubrication +6.3,64.8,53.4,18.7,2011.6,low_lubrication +3.1,57.0,37.2,11.6,1941.7,normal +5.8,57.8,61.1,21.7,4088.6,low_lubrication +1.5,64.7,68.4,26.7,680.1,normal +2.5,54.4,87.1,32.6,70.7,normal +4.8,71.2,51.5,15.6,2863.0,low_lubrication +2.6,89.0,42.7,21.6,2836.4,overheating +4.9,96.7,64.7,28.9,1309.8,overheating +1.5,54.8,74.8,28.2,383.7,normal +10.1,74.3,55.6,21.8,2352.8,bearing_wear +2.2,57.4,65.7,22.7,597.0,normal +8.7,79.3,75.9,25.3,3232.8,bearing_wear +8.2,81.6,59.2,22.9,2609.3,bearing_wear +2.0,59.7,42.1,17.7,794.3,normal +4.5,95.6,66.4,33.1,2746.4,overheating +8.7,69.6,89.9,30.3,1854.5,bearing_wear +8.7,68.4,52.0,19.6,2828.0,bearing_wear +2.6,59.1,39.2,11.4,2187.1,normal +2.9,60.0,72.1,25.5,2136.0,normal +9.9,73.4,30.3,9.2,2218.7,bearing_wear +0.5,47.2,75.5,22.9,624.6,normal +2.5,70.2,62.2,20.5,2232.3,normal +7.0,77.8,60.2,20.7,1559.8,bearing_wear +2.2,105.8,38.7,20.3,924.0,overheating +4.8,72.3,45.5,16.7,3412.2,bearing_wear +6.9,67.3,72.3,24.9,3167.5,low_lubrication +3.3,87.6,84.7,36.9,3139.0,overheating +7.5,74.5,36.3,15.2,1348.0,bearing_wear +6.9,70.2,60.8,18.0,2435.2,low_lubrication +0.3,53.8,61.9,24.7,198.8,normal +8.4,62.8,59.5,22.0,3390.3,bearing_wear +2.3,59.3,61.5,22.3,1480.4,normal +7.7,70.9,87.9,31.5,2930.0,bearing_wear +6.1,83.1,59.5,22.0,1223.9,bearing_wear +3.1,66.6,65.2,21.1,211.2,normal +5.3,58.2,57.8,22.1,2387.3,low_lubrication +5.3,60.8,38.3,14.5,1791.4,bearing_wear +5.3,88.9,74.8,32.6,737.2,overheating +2.5,77.5,67.8,24.4,1026.1,normal +5.8,65.7,58.3,22.4,3641.8,low_lubrication +1.5,61.8,47.6,18.5,2072.9,normal +5.3,66.7,68.9,25.0,3546.0,low_lubrication +3.8,98.4,51.5,21.9,1033.0,overheating +11.7,72.0,85.6,30.4,2379.0,bearing_wear +0.8,58.9,78.1,28.2,419.8,normal +2.9,60.5,32.6,8.1,574.9,normal +2.4,58.7,30.7,8.9,1697.9,normal +4.5,64.1,36.6,14.2,2860.9,low_lubrication +3.5,65.9,61.4,21.3,1594.5,normal +3.2,88.5,78.4,33.2,2060.6,overheating +5.9,61.1,69.9,25.3,3060.8,low_lubrication +12.2,73.1,46.5,14.6,2412.2,bearing_wear +1.7,64.6,77.0,27.4,2350.9,normal +6.4,62.3,58.6,21.8,3523.4,low_lubrication +2.8,62.7,41.9,11.4,1469.6,normal +3.9,101.2,84.5,35.7,1577.6,overheating +3.6,65.3,38.4,13.6,453.8,normal +5.5,53.9,89.8,31.2,3403.8,low_lubrication +0.6,98.8,53.2,22.6,2448.9,overheating +2.3,56.2,67.8,23.7,1905.2,normal +5.0,89.0,41.9,19.1,834.1,overheating +4.6,92.2,42.4,23.5,2564.4,overheating +6.6,75.2,45.8,15.2,3006.0,bearing_wear +5.8,69.4,85.3,33.3,3581.6,low_lubrication +3.5,81.5,84.9,34.0,3375.5,overheating +2.1,72.0,42.1,12.0,31.0,normal +4.5,78.8,47.1,23.1,3141.3,overheating +3.1,79.3,73.6,29.7,2656.4,overheating +5.4,65.1,85.0,32.9,2240.3,low_lubrication +4.2,87.4,65.2,29.0,2928.6,overheating +3.8,61.7,68.9,21.1,86.2,normal +2.6,85.2,56.7,30.6,1123.9,overheating +4.1,85.6,35.1,14.5,2012.9,overheating +5.9,64.8,62.8,24.9,3537.4,low_lubrication +10.0,71.6,85.1,33.1,1919.0,bearing_wear +2.2,63.4,39.9,14.8,1668.5,normal +2.9,55.7,37.8,13.6,1480.7,normal +5.8,63.7,46.4,16.7,3788.2,low_lubrication +3.2,92.0,61.9,26.7,854.9,overheating +1.8,61.4,36.3,13.3,713.3,normal +4.5,93.4,67.6,29.3,2664.0,overheating +7.3,71.5,74.1,28.3,3651.6,low_lubrication +7.6,83.0,81.3,27.8,2566.9,bearing_wear +2.0,63.9,71.8,25.1,1371.5,normal +4.3,72.9,61.3,19.8,3047.3,low_lubrication +1.5,63.8,75.7,25.3,806.2,normal +2.4,59.7,57.7,21.4,581.8,normal +1.2,50.6,49.9,20.2,2066.9,normal +3.2,64.4,70.9,22.3,2710.1,low_lubrication +5.3,66.6,86.5,30.1,2360.3,low_lubrication +7.7,68.9,41.2,16.3,1862.1,bearing_wear +2.3,67.6,46.7,16.6,2328.1,normal +5.8,59.6,42.1,15.6,3872.9,low_lubrication +6.6,84.0,38.8,15.3,3297.3,bearing_wear +5.2,76.9,81.8,32.8,3397.6,low_lubrication +4.0,95.1,66.7,35.1,970.4,overheating +3.2,67.2,88.2,33.2,1550.4,normal +6.1,68.2,67.4,23.1,2578.6,low_lubrication +3.6,94.4,46.4,22.6,2830.8,overheating +2.9,58.4,44.9,18.8,2001.4,normal +7.1,67.9,49.8,16.6,1235.4,bearing_wear +4.8,59.5,69.2,24.6,3006.1,low_lubrication +6.2,68.5,51.4,18.4,3286.7,bearing_wear +4.1,58.1,43.8,15.8,2219.4,normal +4.6,103.1,75.1,32.8,1845.9,overheating +3.0,96.7,81.8,36.3,2745.7,overheating +3.5,60.3,54.9,18.7,1358.3,normal +5.1,63.3,38.9,12.3,2317.2,low_lubrication +3.1,93.0,38.4,19.6,2717.7,overheating +8.3,87.6,32.3,10.3,2975.4,bearing_wear +8.1,66.8,88.3,31.2,2674.4,bearing_wear +3.8,42.3,38.8,16.0,1678.5,normal +5.9,71.0,86.5,31.4,2944.7,low_lubrication +4.8,69.4,66.6,20.9,3492.9,low_lubrication +5.9,68.5,45.4,21.2,3382.2,bearing_wear +4.0,68.1,88.1,33.3,2945.7,low_lubrication +4.4,64.7,79.7,32.2,2098.5,low_lubrication +4.5,68.6,36.8,14.1,3144.8,low_lubrication +8.9,65.0,75.0,27.3,3225.2,bearing_wear +0.7,63.0,70.2,21.2,773.9,normal +7.9,73.6,60.4,17.5,3043.0,bearing_wear +1.5,90.7,85.1,37.2,505.6,overheating +4.5,96.3,50.9,21.7,1746.2,overheating +2.4,52.2,46.7,16.6,1185.6,normal +5.4,67.0,53.8,19.9,2174.2,low_lubrication +6.2,65.6,33.9,12.8,3913.5,low_lubrication +4.9,89.6,41.4,20.4,2073.7,overheating +2.7,59.8,69.9,27.2,2342.7,normal +3.3,89.1,59.4,26.7,1870.6,overheating +8.8,65.9,40.3,18.6,1910.4,bearing_wear +6.2,66.8,51.4,19.1,2490.6,low_lubrication +8.8,68.3,76.6,27.1,2806.0,bearing_wear +12.0,72.2,72.6,25.8,2642.4,bearing_wear +5.4,60.3,71.2,25.7,3003.8,low_lubrication +6.6,68.7,73.6,26.6,1335.6,bearing_wear +5.2,62.0,88.1,33.4,2103.6,low_lubrication +9.3,72.2,46.3,16.0,2923.8,bearing_wear +2.4,57.5,45.2,16.2,1885.2,normal +4.4,62.2,60.4,21.8,3076.7,low_lubrication +3.1,51.5,44.3,17.2,802.1,normal +4.4,70.0,64.1,24.4,3725.6,low_lubrication +4.9,65.4,67.5,23.5,3880.8,low_lubrication +6.0,59.6,58.5,23.6,3583.2,low_lubrication +2.1,55.6,73.8,27.2,821.9,normal +2.5,62.2,85.3,32.0,953.3,normal +1.8,82.3,52.1,23.5,3422.5,overheating +8.7,69.3,83.2,27.0,2734.3,bearing_wear +3.0,103.4,33.6,17.6,511.8,overheating +7.9,74.5,35.6,10.7,2494.2,bearing_wear +3.0,95.2,82.6,37.1,1811.8,overheating +4.1,51.7,39.9,18.2,2189.3,normal +6.2,96.0,85.1,31.8,1338.1,overheating +3.6,59.4,76.5,28.3,2471.1,normal +3.1,89.4,42.3,23.0,1322.1,overheating +4.9,63.3,57.3,21.5,2106.8,low_lubrication +4.9,63.8,66.7,25.3,3150.0,low_lubrication +7.8,74.8,79.6,25.5,2507.9,bearing_wear +3.1,92.8,36.5,19.8,1718.6,overheating +2.4,59.7,33.1,10.8,950.6,normal +3.7,55.6,38.6,14.0,613.4,normal +8.3,75.3,65.8,22.3,2871.9,bearing_wear +8.3,70.0,63.4,20.8,1564.4,bearing_wear +5.5,68.7,71.2,27.1,2637.9,low_lubrication +3.5,98.0,50.5,23.6,1399.9,overheating +3.1,59.9,77.2,29.2,1743.2,normal +6.4,65.7,89.1,32.8,3491.3,low_lubrication +1.5,85.4,60.3,28.9,1978.9,overheating +9.9,76.9,82.7,29.7,1799.3,bearing_wear +2.6,56.3,80.9,25.7,1810.0,normal +5.0,68.3,31.5,12.6,2322.5,low_lubrication +2.0,83.2,43.5,23.0,859.3,overheating +10.1,72.3,42.5,12.0,3482.7,bearing_wear +3.2,100.2,41.2,18.4,2111.2,overheating +3.3,64.4,37.3,12.0,1086.5,normal +2.8,87.9,77.5,35.3,2828.6,overheating +2.5,54.5,52.5,15.9,1252.5,normal +6.8,76.3,70.6,24.1,2629.8,bearing_wear +5.8,74.1,59.5,21.5,2399.7,low_lubrication +6.6,76.6,67.7,28.0,3585.3,low_lubrication +10.1,73.9,85.6,32.0,3083.3,bearing_wear +7.4,58.7,72.0,25.7,2738.0,bearing_wear +1.9,51.7,56.3,20.7,297.6,normal +4.7,68.7,54.4,16.0,4195.1,low_lubrication +7.6,67.4,36.6,10.3,3231.8,bearing_wear +4.1,65.1,85.6,30.1,228.9,normal +2.4,84.8,80.9,36.7,1010.5,overheating +9.0,79.4,66.9,24.5,3417.9,bearing_wear +8.4,79.7,68.4,25.8,1877.6,bearing_wear +8.1,75.0,56.5,17.7,2639.9,bearing_wear +7.4,70.7,75.7,24.5,1867.1,bearing_wear +4.1,97.2,66.4,29.2,2588.0,overheating +4.1,70.6,73.2,26.6,2419.7,low_lubrication +2.7,88.1,54.1,23.1,1216.8,overheating +2.8,85.7,57.1,25.9,630.6,overheating +3.2,51.7,40.4,16.9,2037.2,normal +3.3,91.3,66.0,33.1,911.9,overheating +5.6,59.8,47.7,20.7,3949.6,low_lubrication +5.7,57.2,51.9,16.3,2449.0,low_lubrication +4.7,63.3,68.2,25.2,2312.5,low_lubrication +2.9,62.8,37.9,15.6,2157.3,normal +1.7,43.5,77.0,27.8,1998.8,normal +2.6,85.2,65.3,30.7,2795.1,overheating +3.4,97.3,50.5,24.6,1052.7,overheating +2.3,62.2,56.3,18.0,1579.1,normal +3.8,56.5,68.5,26.3,1993.2,normal +5.7,68.9,76.6,29.8,2326.1,low_lubrication +6.1,65.7,45.5,17.7,4006.5,low_lubrication +0.2,62.4,51.9,18.9,1926.0,normal +5.6,64.7,54.3,17.3,2260.0,low_lubrication +3.5,62.8,83.5,28.0,360.4,normal +2.3,98.6,87.2,39.9,1533.4,overheating +2.4,57.7,38.1,9.4,983.6,normal +6.2,70.2,72.0,26.6,2438.3,low_lubrication +1.4,58.1,57.0,22.7,1468.8,normal +2.6,93.1,32.2,16.6,1531.0,overheating +5.5,71.5,74.6,26.0,2189.4,low_lubrication +4.6,60.6,31.6,10.3,3221.7,low_lubrication +3.2,55.7,71.0,26.5,1515.5,normal +5.9,71.2,76.4,26.3,3482.6,low_lubrication +1.9,67.1,43.5,16.6,1140.6,normal +3.2,61.9,84.6,30.1,581.4,normal +3.8,58.7,66.4,22.3,1118.2,normal +3.1,100.5,33.4,13.9,936.8,overheating +3.1,81.1,55.8,23.8,1269.6,overheating +2.7,66.0,56.6,20.4,401.0,normal +6.8,64.9,50.1,21.2,3440.6,low_lubrication +3.0,56.7,74.3,27.2,1437.2,normal +7.6,79.0,77.7,30.8,1993.4,bearing_wear +4.5,98.4,53.0,25.9,2737.8,overheating +2.0,56.6,84.6,30.8,1864.2,normal +8.8,74.9,57.3,21.6,2800.6,bearing_wear +5.9,65.0,59.9,20.6,3298.4,low_lubrication +2.3,47.4,47.3,19.8,2174.2,normal +3.8,50.6,68.1,22.1,1621.9,normal +2.6,67.4,48.9,18.1,2204.8,normal +5.7,65.4,81.8,31.9,3352.2,low_lubrication +6.6,75.4,61.1,21.9,2386.5,bearing_wear +3.5,86.8,77.4,29.0,2002.2,overheating +2.8,73.8,81.1,28.2,2231.2,normal +3.9,91.9,78.5,33.2,3284.7,overheating +4.6,80.5,63.5,21.3,1556.4,bearing_wear +3.0,61.8,49.5,19.8,1318.5,normal +5.6,68.6,34.9,11.8,3591.2,low_lubrication +7.1,75.1,36.9,12.0,1360.6,bearing_wear +8.9,70.5,73.0,25.2,2707.9,bearing_wear +3.7,65.5,65.2,23.6,603.2,normal +3.6,90.8,63.6,29.1,2907.9,overheating +4.0,94.7,38.3,18.6,949.2,overheating +5.2,67.5,83.7,28.7,2335.7,low_lubrication +4.2,65.8,44.0,14.6,84.7,normal +5.3,61.0,34.5,10.9,2132.9,low_lubrication +7.1,63.7,32.2,11.2,1484.9,bearing_wear +2.2,60.2,71.2,23.9,719.5,normal +7.9,76.5,47.5,17.2,3489.7,bearing_wear +1.7,64.8,35.4,15.6,298.6,normal +2.2,76.9,48.5,19.0,586.7,normal +4.2,71.6,36.2,9.7,3802.5,low_lubrication +5.2,63.8,38.8,16.9,3601.3,low_lubrication +2.8,49.3,83.9,28.3,1566.9,normal +4.8,67.5,65.4,24.2,2790.2,low_lubrication +9.0,69.4,56.7,24.0,1334.9,bearing_wear +9.5,78.3,63.9,22.1,3094.5,bearing_wear +2.5,72.6,68.5,24.6,1961.8,normal +8.4,73.0,33.4,11.0,2583.8,bearing_wear +2.9,64.6,72.6,26.3,50.7,normal +9.3,67.0,64.2,25.5,3077.1,bearing_wear +2.0,83.4,51.0,24.2,2673.4,overheating +3.0,90.6,72.8,40.2,1680.8,overheating +2.9,56.6,62.1,22.9,563.2,normal +2.7,46.5,56.2,22.1,1483.6,normal +4.4,66.2,82.3,26.7,3853.9,low_lubrication +2.3,59.5,49.0,18.5,2498.5,normal +1.7,98.7,71.0,30.1,574.0,overheating +2.8,89.2,65.9,30.6,2445.1,overheating +3.6,61.2,39.2,13.5,2188.2,normal +7.2,67.4,84.3,29.5,1340.2,bearing_wear +2.2,93.5,81.3,34.2,1066.1,overheating +3.5,70.8,58.4,20.7,1683.8,normal +8.9,67.2,82.7,31.9,1648.8,bearing_wear +7.7,68.9,49.7,18.0,2838.7,bearing_wear +3.3,95.3,70.0,33.3,608.2,overheating +3.9,95.9,87.1,37.1,3069.8,overheating +5.3,54.3,61.7,19.1,3694.1,low_lubrication +5.4,66.9,45.7,17.7,2228.5,low_lubrication +5.1,64.0,38.4,12.3,2640.0,bearing_wear +6.2,68.2,54.3,13.8,3998.3,low_lubrication +2.3,65.2,33.3,8.8,28.0,normal +8.8,77.5,85.9,33.8,2271.2,bearing_wear +1.6,52.2,32.7,8.2,1141.9,normal +5.8,66.0,38.1,13.9,3441.0,low_lubrication +5.6,70.1,47.2,19.1,2858.1,bearing_wear +5.4,86.2,45.4,25.3,1294.4,overheating +6.9,64.2,66.6,24.2,2950.3,low_lubrication +3.6,93.5,77.6,34.9,2530.6,overheating +2.5,49.5,70.7,23.9,1302.4,normal +6.5,72.8,39.3,16.5,2100.3,bearing_wear +4.1,66.5,43.4,18.4,3214.4,bearing_wear +3.0,94.4,74.0,32.3,3045.3,overheating +6.1,70.2,32.0,13.1,3244.3,bearing_wear +7.2,75.2,59.0,25.0,3198.4,bearing_wear +4.5,64.8,55.5,19.7,3886.2,low_lubrication +2.7,85.1,34.1,17.2,3380.1,overheating +1.9,88.9,52.7,26.7,1238.7,overheating +9.2,73.3,75.0,26.0,3079.7,bearing_wear +3.4,95.2,52.7,26.2,808.7,overheating +3.6,62.5,41.7,14.4,1603.9,normal +7.2,73.9,67.2,24.1,1872.1,bearing_wear +6.0,72.9,76.7,26.6,3803.4,low_lubrication +2.9,93.0,44.7,21.0,1231.5,overheating +1.9,68.2,77.1,30.9,1229.0,normal +4.5,86.9,57.1,26.3,3099.1,overheating +5.6,60.9,86.6,30.0,3227.1,low_lubrication +5.8,70.8,63.8,18.7,2543.3,bearing_wear +6.5,73.7,79.3,27.2,2553.6,bearing_wear +6.4,64.6,86.0,27.3,4184.8,low_lubrication +6.4,92.6,49.9,20.9,2743.1,bearing_wear +1.0,94.9,60.8,26.7,2172.2,overheating +6.7,68.2,43.8,13.8,4185.9,low_lubrication +5.5,77.2,86.6,29.3,3708.0,low_lubrication +1.1,58.8,35.0,11.4,103.1,normal +8.5,67.0,66.7,23.8,2511.4,bearing_wear +7.1,72.2,53.4,19.2,1202.4,bearing_wear +4.3,62.8,72.7,26.2,2025.5,low_lubrication +3.2,87.6,59.7,24.7,1412.9,overheating +7.3,71.3,87.6,32.3,3460.9,bearing_wear +3.3,81.6,82.9,38.3,2149.5,overheating +4.6,70.0,84.9,30.3,3867.4,low_lubrication +5.8,65.3,75.3,30.2,3303.0,low_lubrication +5.4,66.5,36.9,14.2,2131.2,low_lubrication +9.5,65.8,68.8,23.6,2246.0,bearing_wear +2.8,61.8,31.3,10.1,1741.8,normal +0.8,89.7,75.5,35.0,692.6,overheating +9.3,79.9,73.1,24.8,1621.2,bearing_wear +5.2,70.1,87.7,34.1,3031.2,low_lubrication +4.1,62.0,87.2,29.9,221.8,normal +2.7,56.1,60.3,21.6,612.1,normal +6.6,74.8,32.8,11.1,2595.4,bearing_wear +1.6,66.8,80.6,26.2,2018.6,normal +4.8,62.0,67.3,23.1,4024.1,low_lubrication +4.8,73.5,31.2,17.7,1533.5,overheating +3.3,90.7,69.1,29.1,2109.0,overheating +7.6,75.5,86.6,33.0,1758.4,bearing_wear +2.0,64.8,57.0,20.7,41.2,normal +2.6,63.2,48.3,18.7,373.1,normal +2.8,95.5,52.6,26.5,2486.5,overheating +2.9,74.2,81.4,27.8,1995.9,normal +6.0,71.4,80.6,31.9,2928.3,low_lubrication +2.4,55.1,44.6,18.2,400.8,normal +2.3,60.8,62.9,23.9,402.6,normal +4.7,92.1,73.5,35.3,1865.7,overheating +5.6,66.2,40.6,13.9,2457.0,low_lubrication +3.0,100.3,79.2,35.7,3033.6,overheating +7.3,74.1,59.9,21.1,1769.9,bearing_wear +6.5,80.8,81.8,26.4,3415.8,bearing_wear +8.7,67.6,66.6,22.9,2736.5,bearing_wear +6.3,63.6,75.8,31.8,3854.2,low_lubrication +4.5,84.1,83.4,38.3,2020.3,overheating +3.2,99.2,35.6,19.7,3162.5,overheating +3.3,97.4,89.4,39.0,963.5,overheating +5.0,67.5,34.6,15.5,3000.9,low_lubrication +3.3,65.0,69.5,24.3,187.8,normal +6.1,70.5,34.6,13.3,1308.4,bearing_wear +1.6,63.2,74.7,26.9,309.6,normal +2.6,69.5,32.2,11.2,1918.8,normal +11.3,54.1,61.3,20.3,2344.4,bearing_wear +3.4,75.4,71.4,32.9,3312.0,overheating +5.0,61.7,67.5,25.5,2733.3,low_lubrication +5.2,64.8,87.3,30.9,3144.7,bearing_wear +4.2,90.4,58.2,26.5,1385.4,overheating +5.6,64.9,78.6,27.3,3966.1,low_lubrication +3.1,64.8,65.5,24.6,1695.8,normal +5.9,62.4,81.2,27.4,2428.3,low_lubrication +8.8,75.8,72.1,26.6,2028.9,bearing_wear +6.0,62.3,77.3,31.0,3986.3,low_lubrication +4.8,74.9,43.7,16.1,3130.4,bearing_wear +3.1,57.5,56.2,20.3,1380.6,normal +2.3,99.8,80.4,34.7,3430.3,overheating +6.0,64.4,64.3,22.5,3928.7,low_lubrication +0.6,61.2,66.7,21.8,296.2,normal +6.7,65.3,77.6,26.4,2652.2,low_lubrication +6.6,74.9,62.6,21.3,1233.2,bearing_wear +8.1,78.4,46.7,18.6,2123.3,bearing_wear +7.1,70.2,74.1,28.0,2325.9,bearing_wear +2.9,85.4,77.3,36.1,2050.9,overheating +5.2,74.6,88.5,32.5,3334.8,low_lubrication +4.0,90.6,67.7,28.4,1506.8,overheating +2.5,64.9,55.6,21.1,1180.4,normal +8.6,72.4,68.9,21.8,3179.0,bearing_wear +5.8,63.9,74.6,27.0,3698.5,low_lubrication +5.9,60.7,70.5,23.9,2565.1,low_lubrication +4.8,89.9,38.8,18.5,2191.5,overheating +9.2,74.0,86.0,30.7,2947.2,bearing_wear +7.3,72.7,41.4,14.4,2696.2,bearing_wear +2.5,81.6,83.9,32.8,526.6,overheating +4.0,92.4,52.7,25.6,1310.0,overheating +1.8,86.7,56.0,27.0,1340.3,overheating +5.3,59.2,35.6,11.5,2149.6,low_lubrication +7.4,75.6,81.3,26.4,2404.3,bearing_wear +5.2,63.5,61.7,23.3,3771.7,low_lubrication +4.4,95.7,48.7,23.1,573.7,overheating +8.3,66.7,82.7,30.1,3365.1,bearing_wear +1.9,48.0,35.7,13.3,789.4,normal +3.6,91.6,67.0,29.5,2584.6,overheating +2.5,59.9,82.5,27.0,1428.0,normal +2.4,54.5,42.0,12.4,496.0,normal +10.2,68.2,78.4,30.9,3486.4,bearing_wear +5.3,78.1,78.6,29.1,2699.7,bearing_wear +5.9,63.3,89.9,34.6,2715.3,low_lubrication +3.9,96.7,64.0,28.6,882.0,overheating +1.3,62.2,51.7,16.2,1565.4,normal +5.0,65.7,49.7,15.8,3544.6,low_lubrication +2.3,51.3,76.7,27.7,341.7,normal +3.2,88.0,83.5,34.9,1731.5,overheating +2.7,68.7,48.1,16.2,1408.3,normal +10.0,75.5,89.1,34.4,1721.7,bearing_wear +5.7,63.4,66.1,26.0,2498.1,low_lubrication +4.5,73.7,38.8,12.1,3382.6,low_lubrication +5.6,69.5,78.2,25.3,3271.6,bearing_wear +7.0,72.7,65.9,24.6,3467.0,low_lubrication +9.7,80.3,87.2,35.2,2576.4,bearing_wear +5.3,61.0,54.5,19.9,2418.5,low_lubrication +6.0,71.0,81.4,28.8,3205.3,low_lubrication +5.2,65.7,36.1,11.6,3887.5,low_lubrication +5.0,68.1,46.8,15.8,3481.8,low_lubrication +2.6,71.2,73.6,29.7,1113.1,normal +8.4,86.5,51.9,19.3,3058.8,bearing_wear +2.2,51.1,59.4,20.8,1637.5,normal +4.0,57.4,44.1,17.8,3965.9,low_lubrication +5.2,66.8,69.0,20.6,3773.3,low_lubrication +3.9,71.0,44.7,20.8,2860.1,overheating +3.1,103.7,38.8,17.4,543.4,overheating +8.0,65.0,88.5,37.6,1529.3,bearing_wear +7.5,69.4,52.9,18.7,1753.5,bearing_wear +1.9,89.5,80.7,38.1,3346.8,overheating +2.9,98.7,46.3,22.3,2182.0,overheating +1.9,61.9,86.9,33.0,515.7,normal +3.5,82.6,61.2,26.9,2300.8,overheating +9.6,78.2,47.2,18.5,2992.3,bearing_wear +3.8,65.7,59.2,22.7,4135.9,low_lubrication +3.1,63.0,78.1,27.2,1652.2,normal +2.3,58.7,46.0,13.3,1578.2,normal +6.4,72.7,85.9,31.1,3033.0,bearing_wear +2.1,56.3,49.6,18.1,437.4,normal +3.1,90.7,44.7,20.7,3252.8,overheating +1.5,63.8,42.5,18.1,251.9,normal +3.9,63.5,74.9,26.8,3833.4,low_lubrication +3.5,90.1,63.4,28.4,1025.1,overheating +4.4,64.7,48.6,15.9,3680.4,low_lubrication +2.0,68.0,86.2,30.9,2152.2,normal +4.9,60.9,70.6,27.0,2247.8,low_lubrication +1.0,59.3,39.7,14.6,2091.6,normal +2.7,65.8,77.8,29.2,1437.1,normal +8.4,62.6,84.4,31.7,3245.2,bearing_wear +6.5,75.7,60.4,21.6,1917.1,bearing_wear +3.4,49.9,73.5,27.5,181.1,normal +8.8,70.1,41.7,13.5,3114.3,bearing_wear +3.1,45.5,72.0,23.1,1544.2,normal +8.4,73.6,37.7,14.3,1751.6,bearing_wear +5.8,70.6,87.7,32.9,3357.6,low_lubrication +6.5,71.2,48.6,15.3,3410.7,bearing_wear +7.3,67.0,66.0,22.9,1284.6,bearing_wear +2.1,81.5,69.5,32.5,521.0,overheating +7.9,82.1,60.4,23.4,2977.4,bearing_wear +2.9,86.0,33.3,21.3,1575.5,overheating +7.5,74.0,58.4,21.7,3428.7,bearing_wear +1.8,56.3,52.1,19.1,232.8,normal +5.0,70.7,55.7,16.1,2523.7,low_lubrication +2.2,82.6,87.3,38.9,971.3,overheating +4.5,97.5,48.9,26.3,548.8,overheating +4.2,73.5,71.5,29.0,2067.1,bearing_wear +8.6,74.8,50.5,19.1,2889.2,bearing_wear +7.3,77.9,34.1,13.9,1822.7,bearing_wear +1.3,71.6,33.8,8.5,1203.4,normal +5.5,73.1,61.3,25.4,4180.1,low_lubrication +2.0,55.6,83.2,28.9,913.0,normal +4.9,72.4,32.2,12.9,3809.0,low_lubrication +1.1,53.7,40.7,15.6,1745.5,normal +6.7,71.4,86.2,32.3,2675.8,bearing_wear +3.0,88.2,51.1,24.1,3167.8,overheating +3.7,97.7,47.5,23.9,3387.8,overheating +5.9,57.4,40.7,14.7,2691.6,low_lubrication +6.8,62.0,52.7,19.5,1344.7,bearing_wear +2.9,62.1,76.9,26.9,1840.9,normal +6.6,70.9,69.7,22.2,3224.5,low_lubrication +2.2,52.4,65.6,24.7,404.5,normal +9.4,69.1,80.2,29.5,2948.2,bearing_wear +6.0,66.4,67.1,26.1,2416.8,low_lubrication +3.3,56.3,35.3,10.6,2280.5,normal +5.6,75.0,66.3,23.9,2024.3,bearing_wear +9.7,78.8,58.4,21.3,1442.8,bearing_wear +4.9,84.7,48.0,15.5,1234.9,bearing_wear +2.8,92.3,51.6,25.3,2428.4,overheating +8.7,77.3,75.9,25.9,1505.7,bearing_wear +6.3,72.7,47.5,19.4,1528.6,bearing_wear +3.4,82.3,82.4,35.1,1038.9,overheating +9.2,76.0,77.0,30.9,1776.2,bearing_wear +5.6,68.6,88.3,27.6,3570.4,low_lubrication +8.3,78.1,48.7,14.6,3295.1,bearing_wear +2.0,93.0,60.4,29.0,3124.3,overheating +4.6,63.2,86.6,31.8,3109.8,low_lubrication +4.1,61.1,76.3,27.5,3069.6,low_lubrication +4.6,70.2,75.6,23.6,3535.4,low_lubrication +7.4,57.6,47.5,18.9,2535.6,bearing_wear +2.2,54.1,69.0,28.9,929.3,normal +2.5,84.8,43.5,21.6,1972.4,overheating +5.3,84.3,71.5,33.5,2141.3,overheating +0.8,60.5,57.5,21.5,845.8,normal +5.8,64.8,75.3,29.7,3610.6,low_lubrication +9.2,67.3,72.3,25.8,2938.1,bearing_wear +3.1,88.8,56.2,30.3,1764.4,overheating +1.6,52.0,80.3,28.4,458.3,normal +4.1,111.6,88.4,38.2,2865.6,overheating +2.9,93.3,63.2,29.4,1137.5,overheating +5.4,65.3,56.0,18.9,3304.6,low_lubrication +1.8,66.8,66.4,23.5,1078.3,normal +2.9,56.5,62.4,22.1,1637.8,normal +1.5,63.4,70.1,24.7,2006.8,normal +3.8,100.2,41.5,23.9,2186.7,overheating +3.2,93.8,49.1,24.1,1414.1,overheating +6.7,73.2,75.6,28.6,1682.8,bearing_wear +4.8,54.6,40.3,14.6,3530.2,low_lubrication +4.9,88.6,72.3,36.4,1458.1,overheating +5.4,70.8,31.2,11.8,3468.7,low_lubrication +7.2,69.3,59.0,23.0,3171.7,bearing_wear +3.3,96.6,67.4,30.3,2538.6,overheating +5.9,69.9,44.8,14.2,3200.6,low_lubrication +3.2,97.3,32.7,18.5,2451.6,overheating +2.2,55.2,88.5,27.0,1465.4,normal +4.4,91.0,41.0,19.9,3338.9,overheating +7.7,59.9,82.8,28.5,2455.0,bearing_wear +4.9,87.8,69.1,28.1,1479.9,overheating +4.8,66.9,33.8,16.6,3989.5,low_lubrication +6.4,58.8,85.2,30.2,3770.4,low_lubrication +8.9,73.4,62.6,20.1,1974.1,bearing_wear +4.5,65.5,31.8,12.4,3834.8,low_lubrication +7.6,84.1,50.6,19.7,2515.5,bearing_wear +5.6,66.4,50.5,18.2,2703.1,low_lubrication +2.5,60.7,80.6,30.7,1526.0,normal +6.6,68.2,33.9,13.8,2124.6,low_lubrication +3.1,92.3,34.5,18.0,2148.5,overheating +8.3,75.2,77.5,27.5,1262.8,bearing_wear +9.4,72.1,66.2,24.6,1508.6,bearing_wear +2.2,61.2,85.7,31.7,1079.2,normal +4.7,70.2,73.6,27.3,2366.4,low_lubrication +3.7,97.7,38.1,20.9,1662.8,overheating +7.6,72.7,64.7,21.7,1558.6,bearing_wear +6.0,64.4,71.7,24.3,2788.1,low_lubrication +8.8,73.2,57.9,21.0,1885.1,bearing_wear +3.4,89.1,51.2,23.3,3008.4,overheating +6.0,65.2,48.3,17.1,2889.5,low_lubrication +7.2,75.4,83.0,28.4,1401.5,bearing_wear +5.1,62.2,89.6,36.0,2663.0,low_lubrication +8.0,71.9,37.4,13.7,2657.3,bearing_wear +5.7,57.2,44.1,9.8,2855.0,low_lubrication +1.2,67.7,75.9,26.1,2397.7,normal +5.8,67.3,84.0,25.4,2453.7,low_lubrication +3.7,91.0,54.7,28.3,2243.3,overheating +3.5,64.2,68.0,26.7,2317.4,normal +0.5,58.7,43.8,14.2,1005.9,normal +2.9,94.3,32.4,19.0,915.0,overheating +4.7,52.7,31.5,13.6,2370.4,normal +7.7,70.3,33.5,7.9,2890.2,bearing_wear +2.4,91.7,65.6,28.1,616.6,overheating +3.6,82.1,44.5,19.2,3402.9,overheating +1.1,62.7,55.9,21.9,1685.4,normal +2.7,62.2,41.8,16.6,1056.2,normal +6.8,64.8,38.1,12.2,3660.1,low_lubrication +4.1,50.9,79.9,28.3,2193.7,normal +7.2,77.4,76.8,30.7,2047.8,bearing_wear +3.4,57.6,81.4,27.0,849.3,normal +9.1,71.5,85.1,28.0,3135.3,bearing_wear +6.2,77.9,57.1,21.2,1964.6,bearing_wear +4.6,82.7,65.8,21.9,1714.0,bearing_wear +1.3,58.6,66.7,26.1,947.8,normal +3.4,56.3,81.1,30.4,1485.2,normal +7.1,70.2,41.3,14.3,2830.8,bearing_wear +4.4,84.8,35.3,19.4,525.5,overheating +1.8,61.0,68.2,21.1,1491.8,normal +8.3,67.6,57.1,24.8,1518.3,bearing_wear +5.0,62.2,74.2,30.5,1470.4,normal +3.4,86.5,87.5,38.4,894.5,overheating +5.5,67.6,55.5,20.9,1664.4,bearing_wear +3.9,58.8,38.3,15.1,3339.7,low_lubrication +3.1,91.9,56.3,26.6,2426.8,overheating +5.2,71.6,55.2,21.0,3166.1,low_lubrication +3.8,53.4,32.2,10.0,373.0,normal +5.9,71.2,32.9,10.7,3472.9,low_lubrication +4.1,91.7,37.1,21.6,1892.3,overheating +1.4,95.4,66.1,31.7,3351.0,overheating +5.0,63.1,88.7,30.1,3441.9,low_lubrication +5.0,75.9,45.2,16.7,3311.3,low_lubrication +1.9,79.5,74.2,36.3,2851.7,overheating +9.2,81.4,31.6,11.4,1713.9,bearing_wear +5.6,69.2,77.1,29.8,3176.5,low_lubrication +2.7,51.0,60.2,24.1,1485.7,normal +2.0,67.6,58.0,20.9,615.0,normal +6.1,62.5,75.9,29.3,4195.6,low_lubrication +5.9,66.9,35.9,15.0,1943.9,bearing_wear +3.7,66.7,40.9,16.4,3768.7,low_lubrication +4.2,85.2,41.7,22.3,859.0,overheating +6.3,68.2,74.0,25.8,2370.4,low_lubrication +2.7,62.0,57.3,20.0,135.1,normal +2.2,58.0,65.5,25.1,1721.9,normal +4.0,61.5,50.0,20.3,3955.8,low_lubrication +0.4,50.5,67.6,21.2,2424.8,normal +5.4,73.0,54.2,20.3,4084.3,low_lubrication +1.8,51.9,67.2,25.2,667.3,normal +5.0,88.2,82.9,35.9,2288.5,overheating +2.8,93.4,60.0,27.5,3156.1,overheating +5.1,82.6,80.8,36.4,804.8,overheating +7.1,81.6,55.5,19.4,2932.5,low_lubrication +2.8,89.2,87.6,37.2,3422.6,overheating +5.8,61.9,81.4,27.6,2513.4,low_lubrication +4.0,60.1,71.9,27.1,2387.1,low_lubrication +8.1,88.1,36.9,11.3,1448.9,bearing_wear +2.7,62.4,40.8,14.9,570.8,normal +7.3,89.4,30.0,12.6,1285.7,bearing_wear +7.6,68.0,59.1,21.5,1307.7,bearing_wear +2.4,84.9,66.1,32.7,651.8,overheating +1.8,91.9,63.6,32.6,2416.1,overheating +3.8,96.6,58.6,26.2,1941.1,overheating +10.6,63.2,68.5,23.7,1439.5,bearing_wear +6.3,62.2,74.5,22.1,2034.7,low_lubrication +2.6,72.0,42.1,13.6,1685.4,normal +5.4,64.4,63.8,22.6,2875.4,low_lubrication +5.9,63.0,82.0,30.8,3809.9,low_lubrication +6.2,54.6,71.5,22.0,2390.8,bearing_wear +10.0,77.9,84.4,24.2,1643.3,bearing_wear +2.3,71.5,53.6,19.3,2326.5,normal +5.3,58.6,50.8,15.3,4030.2,low_lubrication +4.0,94.5,37.0,15.2,2962.6,overheating +5.6,76.8,57.3,21.4,1768.4,bearing_wear +5.0,79.9,51.2,22.8,2510.6,bearing_wear +6.5,65.5,51.9,14.4,1296.7,bearing_wear +4.0,57.6,58.8,20.1,112.4,normal +5.4,68.2,88.9,32.1,2918.0,low_lubrication +5.6,72.7,89.9,33.2,3645.6,low_lubrication +8.3,70.9,75.1,25.7,2661.5,bearing_wear +2.8,60.7,35.9,12.3,1280.7,normal +4.0,95.8,88.9,37.7,1756.8,overheating +1.3,56.2,37.1,15.9,1396.7,normal +3.9,65.9,63.2,26.6,4030.8,low_lubrication +7.7,73.1,79.3,28.8,1865.7,bearing_wear +6.1,67.2,84.9,30.6,2143.1,bearing_wear +8.3,82.7,68.5,23.7,3434.6,bearing_wear +2.9,54.6,47.3,15.3,920.6,normal +1.4,69.2,79.4,25.3,2494.2,normal +2.0,94.6,89.5,40.1,1820.9,overheating +1.1,101.7,89.5,35.0,877.6,overheating +5.0,90.2,73.6,33.5,3374.8,overheating +5.0,60.8,69.6,24.1,1847.9,bearing_wear +2.4,79.1,36.9,20.7,1203.9,overheating +2.1,58.7,61.8,22.3,1535.7,normal +6.6,71.1,39.8,18.4,3025.7,bearing_wear +6.4,67.5,53.8,17.4,3415.0,low_lubrication +4.2,94.3,71.6,35.3,2782.8,overheating +5.8,71.9,40.4,13.7,2597.7,bearing_wear +5.5,65.2,40.1,13.2,2959.3,low_lubrication +6.4,67.2,80.1,28.2,2670.8,low_lubrication +8.4,83.3,87.6,32.7,2466.5,bearing_wear +6.4,57.8,50.6,20.9,3610.6,low_lubrication +8.2,91.1,69.0,26.8,1723.4,bearing_wear +6.9,68.4,31.0,13.3,3489.4,bearing_wear +1.1,52.9,49.7,21.0,82.3,normal +1.5,64.5,38.4,15.3,2315.5,normal +4.8,59.6,38.5,14.4,3916.8,low_lubrication +2.3,46.3,30.6,11.6,2424.5,normal +4.3,78.9,89.4,37.8,1298.9,overheating +3.2,89.3,59.4,27.0,2295.5,overheating +5.2,63.1,36.1,12.3,2160.6,low_lubrication +2.6,64.5,43.1,17.7,2307.2,normal +3.2,62.8,76.1,27.3,835.9,normal +1.9,91.4,88.1,37.7,1397.7,overheating +8.2,82.9,76.9,27.5,3133.8,bearing_wear +5.5,61.0,67.8,22.9,2890.7,low_lubrication +9.0,76.5,47.6,20.7,2248.8,bearing_wear +5.5,93.1,57.9,25.2,2106.0,overheating +0.9,49.2,42.9,13.2,529.4,normal +2.7,94.7,68.4,27.8,1932.3,overheating +5.9,68.7,89.2,32.6,1859.9,bearing_wear +2.3,91.9,35.2,17.7,2094.0,overheating +5.2,70.8,57.5,21.6,3090.1,low_lubrication +5.9,68.9,51.6,17.5,3658.3,low_lubrication +8.4,73.4,67.0,23.5,1254.1,bearing_wear +4.8,67.0,55.8,19.1,2260.1,low_lubrication +7.4,58.4,77.4,30.6,3035.0,bearing_wear +2.8,58.8,85.5,31.6,2034.8,normal +4.8,66.1,32.9,11.1,2916.8,low_lubrication +8.2,72.5,49.9,20.4,3345.3,bearing_wear +10.3,80.7,79.2,28.4,3492.5,bearing_wear +1.9,58.2,40.5,14.8,983.3,normal +4.8,65.1,58.6,19.7,2153.5,bearing_wear +1.6,70.2,72.2,29.2,1426.8,normal +8.3,76.4,77.4,30.9,1864.1,bearing_wear +6.1,67.5,88.4,31.3,2463.8,bearing_wear +5.2,74.6,31.2,10.4,3139.7,low_lubrication +1.9,56.1,32.6,11.9,1561.9,normal +4.6,91.5,59.7,25.4,1256.6,overheating +2.4,53.8,61.2,22.1,519.1,normal +4.5,96.7,37.9,19.7,1589.7,overheating +4.6,98.0,30.6,15.0,1422.8,overheating +1.4,62.0,53.1,19.4,835.2,normal +3.2,49.6,69.7,24.6,1232.1,normal +2.3,93.4,35.7,21.5,817.0,overheating +3.1,62.1,44.2,13.7,1607.0,normal +0.4,59.9,46.0,12.7,1877.5,normal +2.3,67.2,35.7,17.3,1621.5,normal +6.8,81.5,66.0,22.7,1332.4,bearing_wear +5.4,68.1,78.4,27.4,3685.0,low_lubrication +4.9,92.1,42.3,21.7,883.4,overheating +3.2,66.1,79.7,30.8,2675.0,low_lubrication +3.4,56.4,33.2,9.7,2176.8,normal +6.2,74.2,34.0,11.5,2474.6,low_lubrication +5.2,62.4,59.8,22.2,3679.8,low_lubrication +4.5,61.4,59.9,20.3,4114.4,low_lubrication +7.2,79.1,30.8,12.2,2917.5,bearing_wear +9.5,75.3,67.1,25.2,2518.1,bearing_wear +2.2,81.8,68.4,30.6,2686.9,overheating +5.4,63.1,57.9,22.3,3891.7,low_lubrication +2.0,58.9,86.8,31.4,122.2,normal +1.0,73.7,88.1,31.8,682.9,normal +6.1,61.8,64.7,20.9,3538.4,low_lubrication +5.8,71.3,39.8,11.3,2254.6,low_lubrication +2.0,59.9,52.0,20.0,2111.0,normal +6.6,72.3,45.2,14.2,2770.7,bearing_wear +5.8,62.0,73.1,27.1,2930.1,low_lubrication +7.4,74.3,71.3,25.6,1540.5,bearing_wear +2.6,84.5,47.0,26.4,1212.0,overheating +5.0,65.2,85.6,31.0,2103.3,low_lubrication +3.3,70.5,54.1,19.6,4152.6,low_lubrication +1.5,64.5,66.0,24.2,1717.1,normal +3.7,89.6,57.6,30.9,2411.3,overheating +5.1,68.1,87.7,32.6,3282.5,low_lubrication +7.0,77.5,75.4,24.3,3302.3,bearing_wear +4.5,86.5,42.1,18.0,2863.3,overheating +3.4,100.3,72.9,31.7,1442.6,overheating +4.4,95.3,69.9,29.6,2746.2,overheating +5.6,66.7,68.4,23.9,2165.1,low_lubrication +2.7,113.8,76.4,31.4,3300.8,overheating +3.0,89.2,45.0,19.1,1695.6,overheating +6.1,56.5,46.9,15.8,3984.3,low_lubrication +3.3,101.7,68.4,28.4,2365.2,overheating +1.5,81.5,64.4,27.7,988.4,overheating +6.2,63.0,39.8,14.4,3710.6,low_lubrication +3.0,52.1,83.6,30.0,1643.3,normal +5.4,61.3,65.6,23.6,2446.0,low_lubrication +5.5,66.7,42.1,16.6,3892.3,low_lubrication +7.0,71.1,55.1,20.0,3034.1,low_lubrication +5.4,76.6,31.7,12.6,1421.7,bearing_wear +5.9,81.7,72.4,26.6,2237.2,bearing_wear +5.0,69.3,70.7,25.0,2026.1,low_lubrication +3.1,58.8,86.8,30.6,317.1,normal +4.2,66.7,76.3,29.4,3364.5,low_lubrication +6.9,76.3,35.0,12.6,2049.6,bearing_wear +3.2,95.0,45.7,21.4,1909.4,overheating +4.1,62.3,85.9,30.3,3321.4,low_lubrication +6.7,62.0,65.7,22.5,3747.3,low_lubrication +3.8,86.4,52.5,27.9,2426.2,overheating +6.9,63.7,37.6,11.5,2809.0,low_lubrication +3.6,92.5,55.5,23.2,1167.1,overheating +6.8,82.8,56.3,22.4,1212.9,bearing_wear +10.0,70.3,35.7,15.1,2223.3,bearing_wear +3.5,61.3,35.1,12.1,2362.0,normal +4.3,51.4,52.1,19.6,2635.8,low_lubrication +2.2,59.9,55.1,22.2,264.9,normal +1.6,56.0,65.0,24.3,432.3,normal +8.2,77.5,72.0,26.4,3084.6,bearing_wear +7.0,63.6,77.3,25.7,4117.1,low_lubrication +0.9,53.7,88.2,30.9,1463.0,normal +2.4,102.3,34.8,17.3,1120.6,overheating +2.8,89.1,84.6,35.0,3071.0,overheating +6.6,73.4,38.1,15.1,2381.6,bearing_wear +3.9,89.9,46.0,22.8,1694.8,overheating +1.9,57.8,67.8,23.2,920.2,normal +4.1,67.5,65.1,24.1,2085.6,low_lubrication +3.2,97.5,88.4,38.9,2742.1,overheating +9.8,76.4,85.5,27.4,2665.3,bearing_wear +2.9,58.5,61.1,20.8,878.1,normal +7.0,67.4,42.4,16.2,3151.3,low_lubrication +6.3,63.9,34.8,12.8,1345.6,bearing_wear +7.1,69.5,39.5,14.3,1760.2,bearing_wear +2.9,66.3,89.4,33.0,1172.8,normal +3.6,75.3,72.7,26.4,2509.1,bearing_wear +5.2,68.8,56.4,20.4,2743.2,bearing_wear +1.7,59.5,34.0,13.7,1847.9,normal +4.3,69.1,45.0,19.0,2753.7,low_lubrication +6.0,69.3,50.4,17.6,2269.5,bearing_wear +8.7,62.6,43.8,12.3,1495.1,bearing_wear +5.3,66.4,33.2,14.4,2017.9,low_lubrication +3.2,82.9,86.6,38.1,851.6,overheating +2.7,63.3,82.8,26.1,1157.0,normal +2.9,59.0,67.0,23.1,1285.1,normal +6.1,59.9,79.1,31.6,3276.5,low_lubrication +5.0,72.0,89.1,33.7,2938.9,low_lubrication +3.4,63.4,81.5,30.1,746.9,normal +1.2,60.4,36.3,16.2,1877.1,normal +8.5,68.9,70.3,28.5,2698.9,bearing_wear +9.8,77.6,32.8,14.0,3394.1,bearing_wear +5.3,72.4,89.5,33.7,1468.2,bearing_wear +10.1,84.4,35.1,14.7,1384.0,bearing_wear +4.0,64.4,88.6,25.7,2541.0,low_lubrication +3.6,92.1,76.2,33.1,580.2,overheating +4.2,92.8,47.7,25.6,958.9,overheating +3.1,85.9,84.3,37.7,1845.1,overheating +10.8,66.9,80.9,25.5,2916.4,bearing_wear +7.5,80.4,46.2,13.4,3337.7,bearing_wear +3.1,96.0,82.2,39.7,1726.5,overheating +8.2,83.2,66.2,23.2,2835.0,bearing_wear +4.0,87.1,72.4,32.3,2687.8,overheating +10.5,74.8,44.9,16.9,2725.4,bearing_wear +2.7,50.3,51.6,12.5,1474.5,normal +7.7,78.3,36.5,14.7,2628.0,bearing_wear +8.3,70.8,50.7,18.8,2969.5,bearing_wear +2.8,65.7,69.9,24.2,676.2,normal +4.6,66.6,71.4,25.3,2841.5,low_lubrication +4.6,75.6,82.7,36.5,2292.4,overheating +5.1,71.2,70.0,23.4,2843.1,low_lubrication +1.5,90.7,61.1,25.5,2650.6,overheating +5.2,100.1,87.8,39.1,2830.6,overheating +6.2,74.8,76.5,26.6,3044.6,low_lubrication +5.3,64.3,64.2,22.2,2682.0,low_lubrication +7.4,66.8,51.3,20.7,3195.4,bearing_wear +5.9,68.8,64.3,22.4,2659.5,low_lubrication +1.6,53.7,50.3,23.2,696.5,normal +2.5,53.6,52.8,20.6,1912.0,normal +3.6,52.7,40.4,15.5,404.7,normal +5.7,68.4,70.4,23.7,3904.3,low_lubrication +5.2,57.8,57.1,18.9,3982.3,low_lubrication +2.6,61.4,69.4,25.0,160.5,normal +4.1,94.1,75.4,32.1,1291.8,overheating +7.9,71.6,39.9,12.6,1708.4,bearing_wear +3.1,87.6,36.0,18.3,1900.1,overheating +5.1,65.0,39.0,13.6,3433.7,low_lubrication +4.8,94.9,64.8,30.6,553.4,overheating +5.9,58.7,81.4,29.5,3340.1,low_lubrication +1.2,58.9,41.4,15.4,226.0,normal +7.6,69.8,89.9,32.2,3731.1,low_lubrication +6.1,65.5,51.7,18.7,2803.7,low_lubrication +4.7,60.5,47.0,15.2,3070.3,low_lubrication +11.7,82.6,46.1,15.0,1886.3,bearing_wear +4.5,54.3,84.4,28.1,2773.0,low_lubrication +2.9,67.1,36.6,17.4,554.9,normal +3.3,91.2,69.6,30.3,779.2,overheating +4.9,70.2,59.5,21.0,2008.2,low_lubrication +2.1,52.9,76.8,25.4,264.4,normal +1.5,57.3,65.9,21.5,983.1,normal +4.6,72.9,52.9,16.1,3547.8,low_lubrication +5.6,57.8,68.8,28.5,2526.0,low_lubrication +7.0,74.1,72.2,22.9,3100.4,bearing_wear +7.9,68.7,85.7,30.2,2534.9,bearing_wear +4.6,81.9,38.0,20.9,1035.9,overheating +2.6,90.6,39.4,21.7,1704.4,overheating +3.8,89.5,59.5,27.2,1597.0,overheating +2.9,100.8,64.2,26.7,1780.7,overheating +2.8,83.5,50.9,25.8,1793.9,overheating +4.7,94.2,90.0,39.5,737.4,overheating +1.6,65.8,45.0,13.3,2430.0,normal +9.3,68.1,75.7,27.3,1904.6,bearing_wear +2.4,93.7,77.3,33.4,1167.9,overheating +6.7,79.3,31.2,15.4,3116.5,bearing_wear +3.5,53.2,66.1,22.8,383.9,normal +3.6,101.5,78.8,35.0,1670.3,overheating +4.3,70.1,63.2,24.5,127.4,normal +2.3,84.1,55.8,24.2,845.1,overheating +2.2,92.6,76.4,33.5,1193.3,overheating +4.5,80.5,34.0,19.3,3002.0,overheating +4.9,74.1,50.1,24.0,2752.1,overheating +2.1,50.6,81.2,30.6,1246.5,normal +8.6,74.9,70.7,28.7,3138.4,bearing_wear +3.7,100.0,43.8,25.2,1784.6,overheating +2.8,62.5,69.6,22.3,9.7,normal +8.3,75.6,71.6,26.0,1343.9,bearing_wear +5.3,54.4,62.1,21.6,2157.4,low_lubrication +9.0,62.0,33.4,8.5,2929.9,bearing_wear +9.4,71.1,69.4,29.0,3152.9,bearing_wear +3.7,51.3,47.5,17.0,1515.6,normal +4.9,75.5,59.4,19.5,2553.2,low_lubrication +2.1,90.9,38.2,17.8,702.7,overheating +5.0,56.8,31.9,14.5,2650.9,low_lubrication +1.6,55.1,89.2,34.7,1733.8,normal +1.4,63.9,38.4,11.6,1413.7,normal +11.1,75.3,56.9,17.4,2435.5,bearing_wear +2.8,92.6,58.2,25.1,2195.2,overheating +4.9,101.3,58.7,24.9,2948.6,overheating +4.5,64.6,63.2,21.1,1827.6,normal +6.4,71.5,62.5,22.3,2528.7,low_lubrication +3.0,67.9,38.2,14.0,1688.4,normal +9.8,74.4,68.8,24.3,2936.4,bearing_wear +9.3,73.7,33.8,9.5,1620.3,bearing_wear +5.2,66.2,30.8,10.6,2587.3,low_lubrication +5.1,68.4,52.7,15.0,2760.1,low_lubrication +2.8,88.6,71.6,33.0,2480.2,overheating +11.8,67.9,46.0,14.8,1774.0,bearing_wear +6.3,81.2,87.6,33.0,3003.1,low_lubrication +5.0,58.9,82.4,32.0,2829.8,low_lubrication +4.0,66.8,50.6,17.7,3248.0,low_lubrication +9.6,64.0,77.2,25.1,1780.6,bearing_wear +10.1,78.3,85.0,29.4,1997.5,bearing_wear +4.8,63.9,45.0,15.3,2282.0,low_lubrication +2.8,87.1,50.5,26.5,2448.3,overheating +2.8,84.1,52.9,26.9,1243.2,overheating +5.4,63.0,61.2,22.0,3115.0,low_lubrication +1.7,60.3,38.9,12.8,221.1,normal +3.4,88.9,87.1,36.9,2730.8,overheating +3.1,95.0,45.3,19.6,2741.7,overheating +2.6,53.8,78.5,27.5,738.5,normal +4.4,78.8,81.1,35.6,3258.9,overheating +5.3,92.1,37.7,18.6,907.0,overheating +5.4,94.1,39.0,20.0,973.7,overheating +5.4,68.0,80.2,30.8,2709.3,low_lubrication +11.4,76.3,74.8,24.5,1305.7,bearing_wear +1.7,88.5,71.7,33.0,3439.0,overheating +5.4,66.0,49.4,18.7,2955.8,bearing_wear +3.7,63.6,69.5,25.7,3071.4,low_lubrication +7.4,78.7,68.9,26.0,1965.5,bearing_wear +5.7,64.0,56.4,20.8,3514.8,low_lubrication +9.0,74.5,41.9,15.1,2193.3,bearing_wear +5.0,72.9,69.1,24.2,2639.0,low_lubrication +2.4,62.3,76.6,27.9,523.6,normal +2.6,63.7,63.9,25.1,338.1,normal +2.6,96.2,56.5,28.1,1343.8,overheating +7.7,66.7,62.5,19.4,1646.4,bearing_wear +8.7,83.6,48.5,12.0,1323.5,bearing_wear +6.1,67.0,71.9,25.1,2702.7,bearing_wear +3.8,88.3,30.2,16.0,1869.5,overheating +3.0,90.5,45.7,17.5,2153.2,overheating +5.2,61.2,75.3,26.1,2311.2,low_lubrication +3.1,59.2,80.5,30.9,611.6,normal +2.4,56.8,58.2,23.9,1775.0,normal +3.5,58.9,48.1,17.1,1996.9,normal +5.4,64.1,74.1,27.8,2735.1,low_lubrication +6.4,69.9,41.0,13.0,3547.3,low_lubrication +5.0,71.4,80.0,32.4,3435.0,low_lubrication +2.6,60.8,52.6,17.0,2183.1,normal +6.8,72.0,80.1,28.4,2505.0,bearing_wear +5.9,71.8,39.6,13.7,3187.9,low_lubrication +3.4,88.9,31.5,18.8,606.4,overheating +2.2,86.6,55.8,28.2,1271.2,overheating +6.8,80.1,80.3,27.7,2491.4,bearing_wear +7.0,74.3,48.9,15.6,2822.1,bearing_wear +2.5,95.3,70.1,31.1,3456.5,overheating +3.3,101.0,55.5,23.5,987.1,overheating +9.8,70.9,49.7,17.3,1247.5,bearing_wear +5.5,71.1,62.6,26.4,2337.5,low_lubrication +4.9,67.4,88.5,32.2,3631.4,low_lubrication +6.4,58.7,60.0,25.9,2644.1,low_lubrication +7.7,62.5,34.2,11.7,3483.2,low_lubrication +3.1,94.9,37.8,20.7,1442.4,overheating +8.5,67.3,85.0,31.2,2632.2,bearing_wear +11.0,75.5,86.3,29.8,1348.0,bearing_wear +4.0,104.6,73.9,31.8,633.2,overheating +4.4,90.8,73.4,31.6,2756.0,overheating +6.2,59.3,52.1,17.9,3475.6,low_lubrication +4.5,67.4,70.1,27.6,2996.4,low_lubrication +2.9,61.4,55.1,25.2,1849.6,normal +3.5,86.4,56.5,27.4,771.0,overheating +3.4,65.4,59.0,22.7,245.2,normal +7.8,76.5,68.7,25.9,1807.6,bearing_wear +6.3,68.0,42.7,14.0,3396.9,low_lubrication +2.4,98.3,62.0,25.3,3346.3,overheating +0.7,100.7,39.4,18.2,1672.9,overheating +8.8,77.7,72.4,25.5,1304.8,bearing_wear +3.2,91.1,75.6,31.3,1999.1,overheating +9.0,80.0,57.1,17.9,1896.3,bearing_wear +7.9,74.8,53.2,17.4,3093.6,bearing_wear +6.7,67.8,51.6,22.9,2994.0,low_lubrication +4.2,63.1,66.3,20.4,3628.2,low_lubrication +5.4,74.9,35.1,8.7,3416.3,low_lubrication +3.9,68.5,33.1,10.7,3120.1,low_lubrication +3.2,88.8,77.8,32.4,1170.1,overheating +8.0,79.3,39.3,13.8,3100.4,bearing_wear +7.0,71.5,71.7,23.9,2800.9,bearing_wear +8.1,74.9,49.2,16.7,3263.0,bearing_wear +3.8,73.6,59.9,22.2,276.9,normal +6.1,62.7,69.4,25.7,4086.9,low_lubrication +1.4,71.0,77.0,21.6,990.2,normal +3.0,90.8,83.3,33.8,1730.8,overheating +1.3,51.7,43.6,14.8,1835.0,normal +5.9,70.6,34.4,10.0,3443.8,low_lubrication +5.1,61.9,51.9,17.6,2971.4,low_lubrication +9.2,73.6,78.7,29.1,3164.5,bearing_wear +2.5,49.4,85.5,30.4,1391.2,normal +2.2,90.6,82.8,39.0,2894.2,overheating +5.8,66.7,71.8,26.0,2522.0,low_lubrication +8.2,71.9,62.8,24.2,1397.6,bearing_wear +1.6,52.4,57.5,20.8,1847.7,normal +4.8,76.8,82.0,35.2,3342.9,overheating +5.8,65.7,58.2,19.3,2119.6,low_lubrication +5.0,68.1,63.6,27.0,4121.6,low_lubrication +5.1,87.8,31.7,19.1,1933.8,overheating +1.7,59.5,84.4,28.6,1534.0,normal +2.0,65.8,64.3,23.0,2353.4,normal +5.8,68.3,68.6,25.3,3311.3,low_lubrication +4.6,86.0,75.0,34.2,2647.0,overheating +2.9,61.9,58.0,25.5,258.3,normal +2.7,50.6,78.3,29.5,985.2,normal +3.3,81.4,61.5,29.8,2079.9,overheating +5.4,67.9,33.3,16.1,3275.5,low_lubrication +2.5,51.5,69.8,29.5,2115.9,normal +10.3,70.2,78.7,25.0,1499.1,bearing_wear +2.7,104.6,71.2,32.3,2711.1,overheating +7.5,64.9,68.6,27.9,2915.5,bearing_wear +3.3,69.1,37.0,15.8,963.5,normal +8.4,72.5,48.2,14.8,2440.2,bearing_wear +10.3,75.2,45.4,13.9,2764.1,bearing_wear +11.6,68.9,63.7,22.8,1605.7,bearing_wear +2.0,100.7,80.0,34.9,882.3,overheating +2.8,59.3,80.2,25.4,1709.6,normal +8.6,75.0,68.9,23.2,1803.9,bearing_wear +4.1,69.9,64.1,23.0,2971.2,low_lubrication +5.7,67.2,70.7,23.1,2864.9,low_lubrication +5.4,84.9,32.2,17.6,2516.8,overheating +8.4,70.9,44.8,14.4,2354.9,bearing_wear +2.7,98.9,81.4,37.4,1682.1,overheating +3.9,75.5,78.2,28.4,1486.2,bearing_wear +5.4,63.2,79.5,28.4,2758.4,low_lubrication +2.4,66.7,68.0,23.8,785.4,normal +5.3,70.1,41.0,14.9,3019.6,bearing_wear +3.0,88.3,66.1,27.8,2451.9,overheating +6.2,71.4,44.7,16.4,2891.3,low_lubrication +7.9,76.5,46.8,16.1,2162.6,bearing_wear +5.6,56.8,47.7,15.1,3851.6,low_lubrication +7.1,66.8,87.9,31.1,2944.7,low_lubrication +10.1,85.7,57.2,22.5,3064.3,bearing_wear +5.4,61.9,53.5,18.8,3107.1,low_lubrication +4.8,68.5,85.0,30.5,3949.3,low_lubrication +5.5,65.1,83.1,26.8,2391.6,low_lubrication +1.1,61.8,88.9,27.1,353.2,normal +3.5,94.4,61.4,25.8,1364.1,overheating +5.3,61.2,73.1,24.5,3286.6,low_lubrication +2.1,87.3,59.9,28.7,1281.0,overheating +2.1,59.7,48.7,13.3,1115.6,normal +3.9,92.5,49.6,24.5,2103.2,overheating +0.6,68.4,66.2,23.7,1179.7,normal +1.7,53.1,69.0,23.9,799.2,normal +5.7,64.6,76.9,28.2,2085.1,low_lubrication +5.9,55.5,47.6,19.0,3339.4,low_lubrication +2.2,57.9,71.1,25.8,901.4,normal +0.8,60.4,30.4,10.5,1855.8,normal +2.8,54.9,31.1,11.5,2014.5,normal +8.5,72.9,32.0,7.3,1504.6,bearing_wear +8.3,75.5,65.9,22.0,1459.2,bearing_wear +4.2,67.7,67.6,21.1,3160.3,low_lubrication +9.1,72.4,77.5,27.8,3220.3,bearing_wear +5.2,63.2,76.4,27.0,2337.6,low_lubrication +5.0,75.2,34.4,15.3,3254.6,bearing_wear +2.5,58.5,86.2,31.1,1028.0,normal +6.9,85.7,73.4,27.6,3456.6,bearing_wear +6.7,74.7,60.1,17.8,2416.0,bearing_wear +9.6,72.0,58.8,21.9,2855.0,bearing_wear +6.8,66.1,38.0,12.0,2529.5,bearing_wear +6.6,74.6,79.2,29.2,2910.9,bearing_wear +8.3,78.1,63.2,23.0,3035.2,bearing_wear +9.2,81.3,40.2,9.8,2464.1,bearing_wear +5.3,68.6,77.1,25.3,2853.2,low_lubrication +4.0,87.7,70.6,31.3,2309.4,overheating +6.2,67.6,82.2,31.0,3389.3,low_lubrication +3.7,56.5,42.0,18.2,482.6,normal +4.1,72.9,85.4,26.4,2367.7,low_lubrication +1.6,88.2,55.7,26.4,599.8,overheating +5.9,66.3,32.1,11.1,2773.2,low_lubrication +1.2,45.9,89.9,32.1,1624.6,normal +4.7,84.1,33.4,9.6,2292.5,bearing_wear +5.2,64.3,61.2,22.7,3561.6,low_lubrication +7.6,69.3,55.9,18.7,1858.9,bearing_wear +5.3,63.0,36.8,13.6,2916.8,low_lubrication +4.1,101.6,37.5,17.2,767.6,overheating +5.2,57.3,68.7,26.0,2863.4,low_lubrication +4.4,62.8,40.5,13.3,3725.8,low_lubrication +3.1,95.8,54.9,25.4,2026.0,overheating +8.7,75.4,34.2,9.6,2521.3,bearing_wear +2.5,67.7,79.9,30.3,2160.3,normal +3.6,88.9,78.3,33.8,1585.8,overheating +8.2,64.7,86.2,30.7,2931.6,bearing_wear +7.3,65.2,50.8,18.6,2748.8,low_lubrication +3.1,85.9,33.6,18.4,1575.9,overheating +5.8,57.2,78.7,29.7,2245.3,low_lubrication +6.1,74.1,63.6,20.7,3148.9,bearing_wear +5.2,62.1,64.2,20.7,2975.0,low_lubrication +2.7,83.7,81.2,36.6,1041.6,overheating +4.3,92.5,46.7,21.7,1941.3,overheating +2.8,102.2,73.2,30.7,3276.1,overheating +1.4,68.9,72.9,27.3,1900.0,normal +10.6,69.0,75.5,26.3,3122.9,bearing_wear +1.7,58.6,67.6,24.6,150.0,normal +8.1,74.5,73.2,27.2,3260.0,bearing_wear +4.6,63.5,51.1,16.0,2745.6,low_lubrication +8.7,60.4,44.7,15.9,3212.3,bearing_wear +5.7,80.7,51.9,17.0,1358.1,bearing_wear +2.3,57.4,40.6,13.1,1092.5,normal +7.6,73.3,63.3,23.5,1310.6,bearing_wear +6.0,72.2,61.0,21.3,1430.2,bearing_wear +2.1,86.3,55.7,27.8,1280.9,overheating +3.6,92.3,70.6,33.9,1353.8,overheating +4.0,53.0,75.8,27.9,878.5,normal +4.8,100.6,87.5,35.3,2893.8,overheating +3.3,93.2,32.2,15.6,2401.5,overheating +3.2,58.7,76.3,28.7,1869.5,normal +5.2,62.4,70.2,25.7,3796.1,low_lubrication +10.0,74.9,76.9,26.8,3305.1,bearing_wear +2.7,98.4,38.9,19.1,515.6,overheating +4.3,86.8,70.2,32.7,2959.9,overheating +4.8,68.5,31.7,11.0,2659.4,low_lubrication +2.7,55.4,76.1,28.7,2073.1,normal +3.1,62.4,48.9,17.9,2842.1,low_lubrication +3.3,92.9,51.9,24.6,2533.0,overheating +5.4,72.8,86.6,32.9,1243.1,bearing_wear +4.0,69.5,57.9,19.7,3412.2,low_lubrication +3.2,87.0,63.4,29.9,2204.0,overheating +4.2,64.1,46.8,16.3,2433.5,low_lubrication +4.6,63.2,82.3,33.3,3613.3,low_lubrication +4.7,91.4,56.7,25.3,3352.4,overheating +4.0,85.6,33.1,17.0,570.9,overheating +7.4,81.5,66.1,23.4,2654.9,bearing_wear +5.9,61.0,62.9,24.9,3271.2,low_lubrication +6.7,75.9,41.7,14.7,2579.8,bearing_wear +4.0,59.2,68.6,26.7,1890.3,normal +5.5,96.2,72.8,33.1,2527.3,overheating +8.5,71.9,63.2,20.5,3167.9,bearing_wear +2.6,51.8,59.6,27.0,151.8,normal +2.9,88.7,56.2,28.7,1531.5,overheating +4.1,85.2,52.9,22.9,3146.1,overheating +1.5,62.3,82.1,29.8,996.6,normal +4.2,69.6,79.0,30.1,2706.9,low_lubrication +4.3,51.7,37.7,16.0,2392.4,normal +3.5,95.0,39.3,20.9,1467.0,overheating +13.3,72.3,63.6,23.3,1720.3,bearing_wear +6.1,63.2,79.5,27.6,2971.3,low_lubrication +7.8,66.7,82.1,30.2,1913.9,bearing_wear +3.4,95.3,65.9,31.7,1880.8,overheating +4.6,88.4,41.5,23.2,2649.9,overheating +7.6,75.0,63.2,21.7,1460.4,bearing_wear +2.9,94.4,68.5,31.6,3194.2,overheating +4.0,97.4,45.8,22.4,2582.9,overheating +9.8,69.0,72.7,28.7,2722.8,bearing_wear +2.4,68.6,47.1,15.4,2467.4,normal +3.7,81.8,33.0,20.0,3238.8,overheating +7.8,68.7,48.4,18.2,2817.5,bearing_wear +5.8,68.9,70.7,27.9,2152.0,bearing_wear +6.8,77.7,32.4,10.4,2332.7,bearing_wear +6.5,73.0,44.6,18.1,1286.4,bearing_wear +2.8,55.1,46.0,18.1,1150.1,normal +4.7,86.6,33.9,19.5,614.1,overheating +4.0,98.8,44.8,21.9,1716.8,overheating +5.4,58.1,42.2,15.4,2378.0,low_lubrication +2.3,96.3,30.0,17.2,2875.3,overheating +3.4,88.9,81.8,34.1,2630.6,overheating +2.3,57.5,65.8,21.1,2101.7,normal +7.2,68.9,32.8,9.2,3303.6,low_lubrication +7.9,66.4,69.5,23.9,2748.0,bearing_wear +2.5,51.6,38.5,16.4,1627.0,normal +7.0,80.6,34.5,12.4,2288.7,bearing_wear +5.6,59.7,44.5,14.3,3861.2,low_lubrication +2.7,57.1,58.2,17.0,1697.6,normal +4.5,91.4,73.2,33.3,2303.2,overheating diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CMP-001_air_compressor.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CMP-001_air_compressor.md new file mode 100644 index 000000000..989cd2f81 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CMP-001_air_compressor.md @@ -0,0 +1,27 @@ +# SOP-CMP-001 Air compressor + +- **Document:** SOP-CMP-001 +- **Revision:** B (effective 2026-02-05) +- **Applies to:** compressor CMP-01 +- **Owner:** Utilities team +- **Related error code:** E-CMP-107 (discharge temperature high) + +## Limits + +- Normal discharge temperature: 80 to 95 °C. +- E-CMP-107 is raised above **100 °C**. The compressor trips at 110 °C. +- Normal working pressure: 7.5 bar. + +## E-CMP-107: discharge temperature high + +1. Check the compressor room. The room temperature must be below 40 °C and the ventilation openings must be free. +2. Complete SOP-SAF-001, and release all stored air pressure before you open any cover. +3. Check the oil level. Top up with oil **CMP-OIL-46** up to the middle of the sight glass. +4. Clean the cooler fins with compressed air from the inside to the outside. +5. If the temperature is still above 100 °C after cleaning, create a high-priority ticket. Do not run the compressor above 105 °C. + +## Routine + +- Drain the condensate every day. +- Replace the oil and the oil filter every **2000 operating hours**. +- Replace the air filter every 4000 operating hours. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNC-001_spindle_vibration.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNC-001_spindle_vibration.md new file mode 100644 index 000000000..e0c07c011 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNC-001_spindle_vibration.md @@ -0,0 +1,42 @@ +# SOP-CNC-001 CNC spindle vibration and bearing replacement + +- **Document:** SOP-CNC-001 +- **Revision:** C (effective 2026-04-15) +- **Applies to:** CNC milling machines (CNC-03, CNC-07) and the lathe LATHE-05 +- **Owner:** Maintenance Engineering +- **Related error code:** E-SPN-214 (spindle vibration high) + +## Vibration limits + +Vibration is measured at the spindle housing as RMS velocity. + +| Level | Vibration (mm/s RMS) | Action | +|-------|----------------------|--------| +| Normal | below 4.5 | None | +| Warning | 4.5 to 7.0 | Create a medium-priority ticket. Inspect within 24 hours. | +| Stop | 7.1 or higher | Stop the machine. Create a **high-priority** ticket. Do not restart until the cause is fixed. | + +Error code E-SPN-214 is raised by the controller when vibration stays above 7.1 mm/s for 10 seconds. + +## First checks (about 20 minutes) + +1. Complete SOP-SAF-001 (Lockout / Tagout). A spindle always needs a second person as safety observer. +2. Check the tool holder and the tool for damage, and check the clamping force. +3. Check that the tool is balanced. An unbalanced tool is the most common cause of a short vibration peak. +4. Read the bearing temperature. A bearing temperature above 75 °C together with high vibration points to bearing wear. + +## Bearing replacement + +Only a **spindle-certified technician** may replace the spindle bearing cartridge. Plan about 4 hours. + +1. Complete SOP-SAF-001 and move the machine to the maintenance position. +2. Remove the spindle cover and disconnect the coolant line and the sensor cable. +3. Remove the old bearing cartridge. Keep the old cartridge for the failure analysis. +4. Clean the housing and fit the new cartridge (part number SPN-BRG-5518). +5. Tighten the cartridge bolts to **45 Nm** in a cross pattern, in two passes (25 Nm, then 45 Nm). +6. Refit the covers. Run the spindle warm-up program for 20 minutes at 50 percent speed. +7. Record the vibration after warm-up. It must be below **3.0 mm/s**. If it is not, call Maintenance Engineering. + +## Records + +Record the machine runtime hours, the vibration before and after, and the part number in the ticket. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNC-002_coolant_system.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNC-002_coolant_system.md new file mode 100644 index 000000000..28efeb5ba --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNC-002_coolant_system.md @@ -0,0 +1,27 @@ +# SOP-CNC-002 CNC coolant system + +- **Document:** SOP-CNC-002 +- **Revision:** B (effective 2025-11-20) +- **Applies to:** CNC-03, CNC-07, LATHE-05 +- **Owner:** Maintenance Engineering +- **Related error code:** E-CLT-042 (coolant level low) + +## Coolant specification + +- Coolant type: water-miscible emulsion, product code CLT-8. +- Mixing ratio: **8 percent** concentrate. Check with the refractometer. The acceptable range is 7 to 9 percent. +- Tank temperature must stay below 35 °C. + +## E-CLT-042: coolant level low + +1. Stop the program at the end of the current cycle. Do not stop a cut in the middle. +2. Complete SOP-SAF-001 if you must open any cover. +3. Fill the tank with premixed coolant up to the MAX mark. Never add pure concentrate or pure water directly to the tank. +4. Check the filter. Replace the filter if the filter differential pressure is above 1.5 bar. +5. Check the lines and the fittings for leaks. Create a ticket if you find a leak. + +## Weekly tasks + +- Measure the concentration and the pH. The target pH is 8.5 to 9.5. +- Remove tramp oil from the surface. +- Replace the whole tank content every 6 months, or when it smells bad. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNV-001_conveyor.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNV-001_conveyor.md new file mode 100644 index 000000000..949a12491 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-CNV-001_conveyor.md @@ -0,0 +1,24 @@ +# SOP-CNV-001 Belt conveyor + +- **Document:** SOP-CNV-001 +- **Revision:** A (effective 2025-09-12) +- **Applies to:** belt conveyor CONV-03 +- **Owner:** Maintenance Engineering +- **Related error code:** E-CNV-009 (motor overload) + +## Motor current + +- Normal motor current: 8 to 12 A. +- E-CNV-009 is raised when the motor current is above **14 A** for 5 seconds. + +## E-CNV-009: motor overload + +1. Complete SOP-SAF-001. +2. Look for a jam, a stuck roller or product on the belt, and clear it. +3. Turn the rollers by hand. Replace any roller that does not turn freely. +4. Check the belt tracking. The belt edge must stay within 10 mm of the centre line. Adjust the tail pulley in steps of a quarter turn. +5. Check the belt tension. The belt may sag by at most 15 mm in the middle of a span. + +## Lubrication + +Grease the drive and tail bearings every **500 operating hours** with grease **NLGI 2**. Use 3 pumps of the grease gun for each bearing. Do not over-grease. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-GEN-001_ticket_priority.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-GEN-001_ticket_priority.md new file mode 100644 index 000000000..ac2d600d6 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-GEN-001_ticket_priority.md @@ -0,0 +1,29 @@ +# SOP-GEN-001 Maintenance tickets and priorities + +- **Document:** SOP-GEN-001 +- **Revision:** E (effective 2026-06-01) +- **Applies to:** all maintenance staff +- **Owner:** Maintenance Manager + +## When to create a ticket + +Create a ticket for every fault, every warning that is not fixed in the same shift, and every repair that needs a part or a specialist. + +## Priority + +| Priority | Meaning | First response | Target fix | +|----------|---------|----------------|------------| +| high | Safety risk, or the machine is stopped and production is blocked | within 1 hour | same day | +| medium | The machine runs but is degraded, or a warning limit is exceeded | within 8 hours | within 3 days | +| low | Cosmetic issue or planned work | within 5 days | next planned stop | + +## A good ticket has + +1. The machine ID (for example CNC-07). +2. A short summary of the symptom, and the error code if there is one. +3. The priority. +4. The measured values (for example vibration in mm/s) and the runtime hours. + +## Escalation + +If a high-priority ticket is not acknowledged within 1 hour, call the on-duty maintenance lead. The number is on the board in the maintenance office. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-HYD-001_hydraulic_press_oil.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-HYD-001_hydraulic_press_oil.md new file mode 100644 index 000000000..bdeeca76d --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-HYD-001_hydraulic_press_oil.md @@ -0,0 +1,29 @@ +# SOP-HYD-001 Hydraulic press oil temperature + +- **Document:** SOP-HYD-001 +- **Revision:** A (effective 2026-01-10) +- **Applies to:** hydraulic presses (PRESS-02) +- **Owner:** Maintenance Engineering +- **Related error code:** E-HYD-031 (oil temperature high) + +## Limits + +| Level | Oil temperature | Action | +|-------|-----------------|--------| +| Normal | below 65 °C | None | +| Warning | 65 to 75 °C | Create a medium-priority ticket. Find the cause within one shift. | +| Shutdown | above 75 °C | The controller stops the press. Do not restart until the oil is below 55 °C. | + +Error code E-HYD-031 is raised at 65 °C. + +## Causes and checks + +1. Complete SOP-SAF-001. Release the hydraulic pressure through the bleed valve. A hydraulic circuit also needs a safety observer. +2. Check the oil cooler. Clean the cooler fins and check that the cooler fan turns. +3. Check the oil level in the sight glass. Top up with oil **ISO VG 46** only. +4. Read the return filter differential pressure. If it is above **2.5 bar**, replace the filter. +5. Check that the press cycle rate is not above the rated 18 strokes per minute. + +## Oil change + +Change the hydraulic oil every **4000 operating hours** or once per year, whichever comes first. A new fill is 120 litres of ISO VG 46. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-ROB-001_welding_robot.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-ROB-001_welding_robot.md new file mode 100644 index 000000000..0eb71f809 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-ROB-001_welding_robot.md @@ -0,0 +1,21 @@ +# SOP-ROB-001 Welding robot torch and wire feed + +- **Document:** SOP-ROB-001 +- **Revision:** B (effective 2026-05-18) +- **Applies to:** welding robot ROB-04 +- **Owner:** Maintenance Engineering +- **Related error code:** E-WLD-120 (wire feed fault) + +## E-WLD-120: wire feed fault + +1. Move the robot to the service position and complete SOP-SAF-001. +2. Check the wire spool. The wire must unroll freely with no tangles. +3. Check the feed rollers. Replace the rollers if the groove is worn. +4. Replace the liner if the wire is hard to pull through by hand. +5. Replace the contact tip. Use only tips of the same diameter as the wire (1.0 mm). + +## Routine + +- Replace the contact tip after every **8 hours of arc time**. +- Clean the nozzle at every shift change and apply anti-spatter spray. +- Check the tool centre point (TCP) once per month. If the deviation is larger than **0.5 mm**, run the TCP calibration and create a ticket. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-SAF-001_lockout_tagout.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-SAF-001_lockout_tagout.md new file mode 100644 index 000000000..740de4eb8 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/data/sops/SOP-SAF-001_lockout_tagout.md @@ -0,0 +1,25 @@ +# SOP-SAF-001 Lockout / Tagout (LOTO) before maintenance + +- **Document:** SOP-SAF-001 +- **Revision:** D (effective 2026-03-01) +- **Applies to:** all machines at Contoso Precision Machining +- **Owner:** Plant Safety Officer + +## Purpose + +No maintenance task may start until the machine is in a zero-energy state. This SOP applies before every task described in any other SOP. + +## Steps + +1. Tell the shift supervisor and the machine operator that you are about to isolate the machine. +2. Stop the machine using the normal stop procedure. Do not use the emergency stop to stop a running machine, except in an emergency. +3. Switch the main isolator to OFF and fit your **personal red lock** and a danger tag with your name and the date. +4. Isolate every other energy source: compressed air (release the stored pressure), hydraulic pressure (open the bleed valve), and stored mechanical energy (lower or block raised parts). +5. **Verify the zero-energy state.** Try to start the machine from the control panel. The machine must not start. Return the controls to OFF. +6. Only the person who fitted a lock may remove it. If that person has left the site, call the Plant Safety Officer. Never cut a lock without written approval. + +## Rules + +- One lock per person. Group work needs a lock box and one lock per person on the box. +- Work on a spindle, a hydraulic circuit or a pressure vessel also needs a second person present as a **safety observer**. +- Keep hands out of any machine area until step 5 has passed. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/fault_classifier.onnx b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/fault_classifier.onnx new file mode 100644 index 000000000..d1e7cf90d Binary files /dev/null and b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/fault_classifier.onnx differ diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/fault_labels.json b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/fault_labels.json new file mode 100644 index 000000000..78248d20b --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/fault_labels.json @@ -0,0 +1,15 @@ +{ + "features": [ + "vibration_mm_s", + "temperature_c", + "load_pct", + "current_a", + "hours_since_service" + ], + "labels": [ + "normal", + "bearing_wear", + "overheating", + "low_lubrication" + ] +} \ No newline at end of file diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/train_fault_classifier.ipynb b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/train_fault_classifier.ipynb new file mode 100644 index 000000000..95d62d598 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/shortcut/train_fault_classifier.ipynb @@ -0,0 +1,147 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Train a machine fault classifier and export it to ONNX\n", + "\n", + "This is the notebook behind `fault_classifier.onnx`. Read it top to bottom, or run it in an Azure Machine Learning notebook.\n", + "\n", + "1. Upload `sensor_history.csv` (from `resources/data`) next to this notebook.\n", + "2. Run all cells.\n", + "3. Download `fault_classifier.onnx` and `fault_labels.json` and copy them to the edge device." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --quiet scikit-learn skl2onnx onnxruntime pandas" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Look at the data\n", + "\n", + "Each row is one sensor snapshot from a machine, labelled with the fault that was found afterwards." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "\n", + "data = pd.read_csv(\"sensor_history.csv\")\n", + "print(data.shape)\n", + "print(data[\"label\"].value_counts())\n", + "data.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Train\n", + "\n", + "A random forest is small, fast and needs no GPU, which makes it a good fit for an edge device." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.ensemble import RandomForestClassifier\n", + "from sklearn.metrics import accuracy_score, confusion_matrix\n", + "from sklearn.model_selection import train_test_split\n", + "\n", + "FEATURES = [\"vibration_mm_s\", \"temperature_c\", \"load_pct\", \"current_a\", \"hours_since_service\"]\n", + "LABELS = [\"normal\", \"bearing_wear\", \"overheating\", \"low_lubrication\"]\n", + "\n", + "x = data[FEATURES].to_numpy(dtype=\"float32\")\n", + "y = data[\"label\"].map(LABELS.index).to_numpy()\n", + "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=1, stratify=y)\n", + "\n", + "model = RandomForestClassifier(n_estimators=40, max_depth=6, random_state=1).fit(x_train, y_train)\n", + "pred = model.predict(x_test)\n", + "print(\"accuracy:\", round(accuracy_score(y_test, pred), 3))\n", + "print(LABELS)\n", + "print(confusion_matrix(y_test, pred))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Export to ONNX\n", + "\n", + "ONNX is a portable model format. The edge device only needs ONNX Runtime, not scikit-learn and not the cloud." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "from skl2onnx import to_onnx\n", + "\n", + "onnx_model = to_onnx(model, x[:1], options={id(model): {\"zipmap\": False}}, target_opset=17)\n", + "with open(\"fault_classifier.onnx\", \"wb\") as f:\n", + " f.write(onnx_model.SerializeToString())\n", + "with open(\"fault_labels.json\", \"w\") as f:\n", + " json.dump({\"features\": FEATURES, \"labels\": LABELS}, f, indent=2)\n", + "print(\"saved fault_classifier.onnx and fault_labels.json\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. Check that ONNX Runtime gives the same answers\n", + "\n", + "This is exactly what runs on the edge device: no scikit-learn, no network." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import onnxruntime as ort\n", + "\n", + "session = ort.InferenceSession(\"fault_classifier.onnx\", providers=[\"CPUExecutionProvider\"])\n", + "onnx_pred = session.run(None, {session.get_inputs()[0].name: x_test})[0]\n", + "print(\"ONNX matches scikit-learn on\", int((onnx_pred == pred).mean() * 100), \"percent of the test rows\")\n", + "\n", + "# A machine with high vibration, high temperature and many hours since service\n", + "sample = np.array([[9.4, 78.0, 82, 31.5, 1840]], dtype=\"float32\")\n", + "print(\"sample ->\", LABELS[int(session.run(None, {session.get_inputs()[0].name: sample})[0][0])])" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/starter/fl.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/starter/fl.py new file mode 100644 index 000000000..4f93ce989 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/starter/fl.py @@ -0,0 +1,68 @@ +"""Small helpers shared by every challenge: find the local Foundry Local endpoint and load models.""" +import json +import os +import shutil +import subprocess +from pathlib import Path + +from openai import OpenAI + +def _find_data(): + """Find resources/data by walking up from this file, so the same helper works in any lab sub-folder.""" + if os.environ.get("LAB_DATA"): + return Path(os.environ["LAB_DATA"]) + for parent in Path(__file__).resolve().parents: + if (parent / "resources" / "data").is_dir(): + return parent / "resources" / "data" + raise SystemExit("Could not find resources/data. Set the LAB_DATA environment variable to its path.") + + +DATA = _find_data() + + +def _foundry(*args, as_json=True): + exe = shutil.which("foundry") + if exe is None: + raise SystemExit("The 'foundry' command was not found. Install Foundry Local first (challenge 1).") + cmd = [exe, *args] + (["-o", "json"] if as_json else []) + out = subprocess.run(cmd, capture_output=True, text=True, encoding="utf-8", errors="replace") + if out.returncode != 0: + raise SystemExit(f"'{' '.join(cmd)}' failed:\n{out.stdout}{out.stderr}") + return json.loads(out.stdout) if as_json else out.stdout + + +def base_url(): + """Return the OpenAI-compatible URL of the local service, starting the service if needed.""" + if os.environ.get("FOUNDRY_LOCAL_URL"): + return os.environ["FOUNDRY_LOCAL_URL"].rstrip("/") + status = _foundry("server", "status") + if not status.get("running"): + _foundry("server", "start", as_json=False) + status = _foundry("server", "status") + return status["webUrls"][0].rstrip("/") + "/v1" + + +def client(): + # The local service ignores the key, but the OpenAI library requires a value. + return OpenAI(base_url=base_url(), api_key="local") + + +SERVED = Path(os.environ.get("FOUNDRY_SERVED_MODELS", Path.home() / ".foundry" / "served_models.json")) + + +def loaded_models(): + # A service started by serve_offline.py lists what it serves in a small file (the CLI cannot see that service). + if os.environ.get("FOUNDRY_LOCAL_URL") and SERVED.exists(): + return {a: {"alias": a, "id": i, "device": "see serve_offline.py", "fileSizeMb": 0} + for a, i in json.loads(SERVED.read_text(encoding="utf-8")).items()} + return {m["alias"]: m for m in _foundry("model", "list", "--loaded")["models"]} + + +def load(alias): + """Load a model (downloading it first if needed) and return the model id the service expects.""" + if alias not in loaded_models(): + if os.environ.get("FOUNDRY_LOCAL_URL"): + raise SystemExit(f"'{alias}' is not served. Start serve_offline.py with this alias.") + _foundry("model", "download", alias, as_json=False) + _foundry("model", "load", alias, as_json=False) + return loaded_models()[alias]["id"] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/starter/requirements.txt b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/starter/requirements.txt new file mode 100644 index 000000000..756c45e2e --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/resources/starter/requirements.txt @@ -0,0 +1,3 @@ +openai>=1.40 +numpy>=1.26 +onnxruntime>=1.18 diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-01/solution-01.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-01/solution-01.md new file mode 100644 index 000000000..a5cfc7792 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-01/solution-01.md @@ -0,0 +1,99 @@ +# Walkthrough Challenge 1 - Get set up + +**[Home](../../Readme.md)** - [Next Challenge Solution](../challenge-02/solution-02.md) + +Duration: 20 minutes + +## Prerequisites + +Please ensure that you successfully verified the [General prerequisites](../../Readme.md#general-prerequisites) before continuing with this challenge. + +### **Task 1: Install Foundry Local** + +Windows (x64 or ARM64): + +```powershell +winget install Microsoft.FoundryLocal +``` + +macOS on Apple silicon: + +```bash +brew tap microsoft/foundrylocal +brew install foundrylocal +``` + +Open a **new** terminal and check the installation: + +```powershell +foundry --version +foundry model list +``` + +`foundry model list` shows the catalog. The *Device* column says where each model runs (NPU, GPU or CPU) and the *Tools* column shows which models support tool calling. + +### **Task 2: Download a model and chat with it** + +```powershell +foundry model download phi-4-mini +foundry run phi-4-mini +``` + +Type a question, then leave the chat with `exit`. The model files are stored in `~/.foundry/cache/models`. + +### **Task 3: Call the model from Python** + +```powershell +cd 10_Offline_Industrial_Copilot +mkdir work +cd work +python -m venv .venv +.venv\Scripts\activate +pip install -r ..\resources\starter\requirements.txt +copy ..\resources\starter\fl.py . +``` + +[fl.py](../code/fl.py) finds the local service, loads a model and returns the **model ID** that the service expects. Then call the model with the normal OpenAI library ([hello.py](../code/hello.py)): + +```python +import fl + +model = fl.load("phi-4-mini") +client = fl.client() +reply = client.chat.completions.create( + model=model, + messages=[{"role": "user", "content": "In one sentence: what does a technician check first when a CNC spindle vibrates?"}], + max_tokens=80, +) +print(reply.choices[0].message.content) +``` + +Expected: one sentence about the spindle, in 1 to 3 seconds on a laptop with a GPU or NPU. + +### **Task 4: Find the address of the local service** + +```powershell +foundry server status +``` + +The address is `http://127.0.0.1:`. `127.0.0.1` is the **loopback** address: only programs on the same computer can reach it, so no other device and no one on the internet can call your model. The port is chosen when the service starts. `foundry server start --port 5273` pins it, and the `FOUNDRY_LOCAL_URL` environment variable tells `fl.py` where to look. + +You successfully completed challenge 1! 🚀🚀🚀 + +## Coach notes + +- **Pre-cache the models. Verified on the VM.** Sizes on the VM (CPU variants): `phi-4-mini` 4.8 GB, `qwen2.5-1.5b` 1.8 GB, `qwen3-1.7b` 1.3 GB, `qwen3-embedding-0.6b` 0.5 GB. Downloading `phi-4-mini`, `qwen2.5-1.5b` and the embedding model took 92 s, 33 s and 7 s, together 7.6 GB. To build an image with the models already in it: + 1. Download the models once with `foundry model download `. + 2. Copy the cache folder (`foundry cache location` shows it, `~/.foundry/cache/models`) to a shared folder. `robocopy` copied 7.6 GB in 71 s. + 3. Point the service at that folder: `foundry config set cache-directory C:\foundry-models --force`. The service restarts by itself. + + After this, `foundry model list --cached` shows the models and `foundry model download phi-4-mini` finishes in 0.1 s. **Always pass `--force`** in a script: without it `foundry config set cache-directory` and `foundry cache cd` ask for a confirmation and **wait for ever** when no one is at the keyboard. *Not tested:* copying the cache to a different machine. +- **The service needs the model ID, not the alias.** A raw REST call with `phi-4-mini` returns *404 Model not found*. The service expects an ID such as `Phi-4-mini-instruct-generic-gpu:5`. `fl.load()` looks it up with `foundry model list --loaded -o json`. The OpenAI library and the CLI hide this until a team writes its own REST call. +- **Foundry Local picks the variant for the hardware.** The same alias is a GPU variant on one laptop, an NPU variant on a Snapdragon laptop and a CPU variant on the VM. This is why the numbers in challenge 2 differ between teams. +- **Linux is not supported by the command-line tool.** Foundry Local has a Linux SDK, but not the `foundry` command that this MicroHack uses. Use Windows or macOS. +- **The VM from the lab automation. Verified.** It is Windows 11 with Python 3.12 and the Visual C++ runtime installed by the deployment (6.1 minutes for the whole lab). The participant installs Foundry Local with `winget install Microsoft.FoundryLocal`, which took 11 seconds and gave version 0.11.0. Re-checked on 1 November 2026 with version 0.12.1 (14 seconds): challenges 1 to 6 work unchanged, including the offline run. Two things to know: + - **The Visual C++ runtime is required.** On a fresh Windows image without it, `foundry` installs fine, but the service stops at start-up and every command says `Daemon closed the connection mid-stream`. The log (`~/.foundry/logs`) shows `Unable to load DLL 'foundry_local' or one of its dependencies`. The lab automation installs the runtime. A team that builds its own image needs `winget install Microsoft.VCRedist.2015+.x64`. + - **`winget` is not there for a brand-new user for the first minute or so.** Registering the App Installer once fixes it: `Add-AppxPackage -RegisterByFamilyName -MainPackage Microsoft.DesktopAppInstaller_8wekyb3d8bbwe`. + + The VM has no GPU, so Foundry Local picks the CPU variants. `hello.py` answers in 1.4 to 1.8 s there. A tool-using answer takes 18 to 45 s, so plan more time for the VM teams in challenges 3 and 4. *Not tested:* a participant signing in over Remote Desktop. We used the same user's interactive session on the VM console. +- **Teams that finish early:** stream the answer (`stream=True`) and measure the time to the first token. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-02/solution-02.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-02/solution-02.md new file mode 100644 index 000000000..f29aeea61 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-02/solution-02.md @@ -0,0 +1,74 @@ +# Walkthrough Challenge 2 - Choose your edge model + +[Previous Challenge Solution](../challenge-01/solution-01.md) - **[Home](../../Readme.md)** - [Next Challenge Solution](../challenge-03/solution-03.md) + +Duration: 45 minutes + +## Prerequisites + +Please ensure that you successfully completed [Challenge 1](../../challenges/challenge-01.md) before continuing with this challenge. + +### **Task 1: Choose the candidates** + +```powershell +foundry model list +``` + +Good candidates have a **Tools** mark and are small enough for a shop-floor PC: + +| Model | Size | Why | +|-------|------|-----| +| `phi-4-mini` | 3.7 GB | The suggested starting point. Microsoft model, MIT license. | +| `qwen2.5-1.5b` | 1.1 GB | Much smaller. Runs on the NPU on Snapdragon laptops. | +| `qwen3-1.7b` | 1.4 GB | A "reasoning" model: it thinks before it answers. | + +Models without tool calling (for example `phi-4` and `phi-3.5-mini`) cannot be the agent model. A team that picks one learns this in challenge 3. + +### **Task 2: Run the same questions against each model** + +[benchmark.py](../code/benchmark.py) measures speed, tool calling and answer quality for every alias you give it: + +```powershell +python benchmark.py phi-4-mini qwen2.5-1.5b qwen3-1.7b +``` + +How it measures: + +- **Speed:** streams one answer and times the first token and the tokens per second. +- **Tool calling:** runs the 8 cases in `tool_eval.json` once with `tool_choice="auto"` and once with `"required"`. A case passes only if the model makes **exactly one** call, with the right tool and the right arguments (and no call for the question that needs none). +- **Quality:** gives the model the right SOP and counts how many of the 13 answers contain the expected facts (a simple, objective rubric). + +### **Task 3: Read the results** + +Example from a Snapdragon X Elite laptop (Windows 11 on ARM, 64 GB RAM). **Your numbers will be different**, because the device decides which variant runs. + +| Model | Runs on | Size | First token | Tokens/s | Tools `auto` | Tools `required` | Quality | +|-------|---------|------|-------------|----------|--------------|------------------|---------| +| `phi-4-mini` | GPU | 3.8 GB | 0.7 s | 27 | 1/8 | **7/8** | **13/13** | +| `qwen2.5-1.5b` | NPU | 1.1 GB | 0.2 s | 13 to 51 | 1/8 | 0/8 | 13/13 | +| `qwen3-1.7b` | GPU | 1.4 GB | 0.5 s | 32 | 1/8 | 1/8 | 9/13 | + +What the teams should notice: + +- **`tool_choice="auto"` almost never works with small models.** They describe the call in text, or answer from memory. `"required"` forces a structured call, and that is what the agent in challenge 3 uses. Note that the case without a tool (*"Why is the sky blue?"*) can **never** pass with `"required"`, so 7/8 is the best possible score in that column, and `"auto"` scores 1/8 because it passes only that one case. +- **`qwen2.5-1.5b` on the NPU** is the fastest and smallest, and its answers are as good as those of `phi-4-mini`, but with `"required"` the service returns an error (*HTTP 500, inference failed*). It cannot be the agent model on this hardware. +- **`qwen3-1.7b` calls both tools**, including `create_maintenance_ticket` with an invented summary, when asked only for a status. A model that takes actions nobody asked for is worse than a slow one. Its quality score is low partly because it spends its token budget on thinking. Its thinking text arrives in a separate `reasoning_content` field. +- **Memory:** on a GPU or NPU the model does not show up in the process memory. Look at the dedicated or shared GPU memory (Task Manager, Performance tab) or at the NPU. With all three models loaded the Foundry Local service held about 5.9 GB. + +### **Task 4: Choose and explain** + +A good answer for this scenario: + +> *"We chose `phi-4-mini`: it is the only one of the three that made a correct tool call in 7 of 8 cases while giving a correct answer to every question. It is the largest and the slowest, but 0.7 s to the first token is fine for a technician, and it fits on a normal PC."* + +A different choice is fine **if the team has evidence**. A team that finds a smaller model that passes the tool cases has found something useful. + +You successfully completed challenge 2! 🚀🚀🚀 + +## Coach notes + +- Stress **the target device**. What fits on a shop-floor PC matters more than the best benchmark score. +- **Variants differ.** Ask teams which device each model ran on (`foundry model list --loaded`). A model that fails on the NPU may work on the CPU variant. Teams can pick a variant by its full ID. +- **Do not accept "it felt better"**. Ask for the numbers. +- **Stretch:** a third model or another variant (`foundry model info ` lists them) and see how the trade-off shifts. +- The service rejects non-zero `frequency_penalty` and `presence_penalty` with *HTTP 400*. Teams that try to fix repetition with these will hit it. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-03/solution-03.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-03/solution-03.md new file mode 100644 index 000000000..9ad8ef7a8 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-03/solution-03.md @@ -0,0 +1,117 @@ +# Walkthrough Challenge 3 - Build your first local agent + +[Previous Challenge Solution](../challenge-02/solution-02.md) - **[Home](../../Readme.md)** - [Next Challenge Solution](../challenge-04/solution-04.md) + +Duration: 60 minutes + +## Prerequisites + +Please ensure that you successfully completed [Challenge 2](../../challenges/challenge-02.md) before continuing with this challenge. The reference code uses `phi-4-mini`. + +### **Task 1: Write the tools** + +[tools.py](../code/tools.py) has two plain Python functions and their JSON schemas: + +```python +def get_machine_status(machine_id: str) -> dict: + machines = json.loads((DATA / "machines.json").read_text(encoding="utf-8"))["machines"] + for m in machines: + if m["id"].lower() == machine_id.strip().lower(): + return m + return {"error": f"Unknown machine '{machine_id}'", "known_machines": [m["id"] for m in machines]} + + +def create_maintenance_ticket(machine_id: str, summary: str, priority: str) -> dict: + if priority not in ("low", "medium", "high"): + return {"error": "priority must be low, medium or high"} + ... # append one JSON line to tickets.jsonl and return the ticket +``` + +💡 The tools **validate their own input** and return an error the model can read. A small model sometimes sends a machine ID that does not exist (for a question about the cafeteria it sent `cafeteria_menu`). A tool that returns `{"error": ...}` lets the agent recover. A tool that raises an exception ends the conversation. + +### **Task 2: The system prompt** + +```text +You are a maintenance assistant for technicians at Contoso Precision Machining. +Machine data and tickets are only available through your tools: never guess a machine reading. +After you receive tool results, answer in at most three short sentences and then stop. +``` + +Short and specific works best with small models. "Never guess" matters: without it, `phi-4-mini` answers *"I do not have real-time data"* instead of calling the tool. + +### **Task 3: The tool-call loop** + +The loop in [agent.py](../code/agent.py) does this: + +1. Send the conversation and the tool schemas to the model. +2. If the model asks for a tool, run it and add the result to the conversation. +3. Send the conversation again. Stop when the model answers in text. + +```python +choice = "required" if step == 0 else "none" +reply = client.chat.completions.create( + model=model, messages=messages, tools=schemas, tool_choice=choice, temperature=0, max_tokens=500, +).choices[0].message +``` + +Three details make it work with a small model: + +- **The first step uses `tool_choice="required"`.** With `"auto"` the model usually describes the tool call in words (challenge 2: 1 of 8). With `"required"` it makes a proper call (7 of 8). The next step uses `"none"`, so the model has to write the answer. +- **One tool call per step.** Models like `qwen3-1.7b` call several tools at once, including tools nobody asked for. The loop runs the first call and ignores the rest. +- **A step limit.** `max_steps` guarantees that the loop ends. + +### **Task 4: Test it** + +```powershell +python agent.py "What is the status of machine CNC-07?" +``` + +```text + -> get_machine_status({'machine_id': 'CNC-07'}) +The CNC-07 machine is currently in a FAULT state due to high spindle vibration (E-SPN-214). +The vibration measurement is 9.4 mm/s and the temperature is 78.0°C. Service is needed. +``` + +A question that needs no tool, such as *"Why is the sky blue?"*, still gets a forced tool call in this simple loop. The model then picks the closest tool or answers that it has no data. That is a real limit of `"required"`, and challenge 4 shows one way around it. + +### **Task 5: The confirmation step** + +Writing a ticket changes something, so three things protect the technician. All three are in [agent.py](../code/agent.py). + +**1. The write tool is only offered when the technician asked for it.** A model cannot invent a ticket it is never offered: + +```python +WRITE_INTENT = re.compile(r"\b(?:open|create|log|raise|file|make)\b[^.?!]{0,40}\bticket\b", re.I) +wants_write = bool(WRITE_INTENT.search(question)) +offered = write_only if (wants_write and step == 0) else (schemas if wants_write else read_only) +``` + +A ticket request also always reaches the ticket tool, because it is the only tool offered in that step. Without this rule the same request was sent to `search_sops` instead, in 3 of 3 runs on the laptop and also on the CPU VM. + +**2. A person must approve it:** + +```python +if not approve(name, args): + return "Cancelled: the technician did not approve, so nothing was changed." +``` + +**3. The app reports the result, not the model.** After the ticket tool runs, the app builds the confirmation from the tool result: + +```text + The assistant wants to run create_maintenance_ticket({'machine_id': 'CNC-07', 'priority': 'high', ...}). Allow? [y/N] y +Ticket MT-1001 created for CNC-07 (high priority): Spindle vibration above the stop limit +``` + +💥 **Why the model does not write the confirmation.** In a test where the ticket tool was never called, `phi-4-mini` answered *"A high-priority ticket has been opened for CNC-07"*. No ticket existed. An agent that tells a technician that work is logged when it is not is worse than an agent that fails. So the app writes the confirmation from the tool result, and `TICKET_CLAIM` in `agent.py` replaces any answer that claims a ticket exists when no ticket tool ran with an honest *"No ticket was created"*. + +💡 This is not only for show. In the model comparison, `qwen3-1.7b` created a ticket with an invented summary for a plain status question. **A person in the loop, plus an app that decides when a write is allowed and reports what really happened, is what makes a small model safe enough to give write access.** + +You successfully completed challenge 3! 🚀🚀🚀 + +## Coach notes + +- If a team's model handles tool calls badly, send it back to [challenge 2](../../challenges/challenge-02.md): switch to a model that passes more tool cases, or use `tool_choice="required"`. +- `phi-4-mini` sometimes writes the call as text, for example `<|tool_call|>[{...}]`. The loop in challenge 4 parses that form as well. +- Give teams that are new to function calling the loop skeleton in Task 3 and let them write the tools themselves. +- **Tool choice is fragile.** Which tool the model picks first depends on the wording of the request, on the variant of the model (GPU, NPU or CPU) and, in our runs, even on the day. With all four tools shown, `phi-4-mini` chose the right first tool for only 14 of 23 held-out requests and never chose the fault classifier (0 of 5). [Challenge 5](../challenge-05/solution-05.md) adds a router that shortlists the tool and takes this to 23 of 23. [check_routing.py](../code/check_routing.py) measures it, on any device type you will use. +- **Stretch:** log every tool call and its arguments to a file, so a technician can see afterwards what the agent did. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-04/solution-04.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-04/solution-04.md new file mode 100644 index 000000000..954b965d8 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-04/solution-04.md @@ -0,0 +1,100 @@ +# Walkthrough Challenge 4 - Agentic retrieval over local knowledge + +[Previous Challenge Solution](../challenge-03/solution-03.md) - **[Home](../../Readme.md)** - [Next Challenge Solution](../challenge-05/solution-05.md) + +Duration: 75 minutes + +## Prerequisites + +Please ensure that you successfully completed [Challenge 3](../../challenges/challenge-03.md) before continuing with this challenge. + +### **Task 1: Load an embedding model next to the chat model** + +```powershell +foundry model download qwen3-embedding-0.6b +``` + +`fl.load("qwen3-embedding-0.6b")` loads it. It is 515 MB, produces vectors with 1,024 numbers, and runs on the GPU while `phi-4-mini` runs next to it. Embedding the 25 chunks of the SOPs takes a few seconds. + +### **Task 2: Chunk the SOPs** + +Every SOP has the same shape: a header with the SOP ID and the revision, then sections starting with `##`. [retrieval.py](../code/retrieval.py) makes **one chunk per section**, and puts the SOP title and header in front of every chunk: + +```python +"text": f"{title} ({sop_id})\n{header}\n\n## {section}\n{body}" +``` + +💡 Why the header travels with the chunk: a section called *"Routine"* means nothing alone. With the header it says *"SOP-CMP-001, air compressor"*, so retrieval can match the machine, and the answer can cite the SOP and its revision. + +### **Task 3: Embed and store** + +```python +vectors = embed(client, embed_model, [c["text"] for c in chunks]) # normalised +INDEX.write_text(json.dumps({"chunks": chunks, "vectors": vectors.tolist()})) +``` + +25 vectors do not need a vector database. A JSON file and NumPy dot products are enough. Because the vectors are normalised, `vectors @ query` is the cosine similarity. + +### **Task 4: Retrieval as a tool** + +```python +def search_sops(query: str) -> dict: + queries = decompose(client, chat_model, query) # one query per topic + for q in queries: + for chunk, score in retrieve(client, embed_model, chunks, vectors, q, k=3): + if score >= 0.35: ... # keep it, once + return {"queries": queries, "excerpts": excerpts or "No relevant SOP found."} +``` + +The agent from challenge 3 gets this as a third tool. It **decides** whether a question needs the SOPs: *"What is the status of CNC-07?"* calls the machine tool, *"At what vibration level must a CNC spindle be stopped?"* calls `search_sops`. + +- **`min_score=0.35`** is what makes the agent refuse. A question about the cafeteria menu finds nothing above the threshold. +- **Compound questions.** A small model rewrites *"What torque do I use for the bolts, and what must I do before I start?"* into one short query and loses a topic. `decompose` asks the model, in plain text, to split the question into topics. Each topic gets its own search, and the excerpts are merged. The tool description and the system prompt tell the agent to pass the technician's whole question. + +### **Task 5: Citations** + +Do not trust the small model to cite. In the test it named the source in only 3 of 13 answers. The app attaches the SOPs it really retrieved, and removes sentences that cite an SOP that does not exist: + +```text +Stop the machine if the vibration level reaches 7.1 mm/s RMS or higher. + Sources: SOP-CNC-001 rev C / Vibration limits; SOP-CNC-001 rev C / Records +``` + +`scrub_citations` in [agent.py](../code/agent.py) matters: without it `phi-4-mini` ends answers with *"Please refer to SOP ID 1024 for maintenance procedures"*, an SOP that does not exist. + +### **Task 6: Evaluate** + +```powershell +python evaluate_retrieval.py # always retrieve, then answer +python evaluate_retrieval.py agent # the agent decides +python evaluate_compound.py # questions with two topics +``` + +Reference results with `phi-4-mini` (13 answerable questions and 2 that are not in the SOPs): + +| | Retrieval finds the right SOP | Answer has the expected facts | Right SOP cited | Refuses what it cannot answer | +|---|---|---|---|---| +| Always retrieve | 13/13 | 12/13 | 11/13 | 2/2 | +| Agent with `search_sops` | n/a | 12/13 | 12/13 | 2/2 | + +Questions with two topics (3 questions, 2 runs each): both SOPs found in 6 of 6 runs, all expected facts in 4 of 6, and two runs in six did not split the question into two topics. The answers are not identical between runs, so a difference of one is noise. + +💡 Retrieval is rarely the problem here: the right SOP is found every time. The misses are in what the small model **does with** the excerpts, for example it quotes the trip temperature (110 °C) instead of the alarm temperature (100 °C). That is the kind of thing a team can improve. + +### **Task 7: Change one thing and measure it** + +Ideas that teams try, and what to watch: + +- **Chunk size:** smaller chunks (one step per chunk) improve precision but can lose context. +- **`k` and `min_score`:** more excerpts help recall and hurt precision. A higher threshold refuses more often, also when it should not. +- **The prompt:** *"quote the number from the excerpt"* often fixes the wrong-number misses. +- **A bigger chat model:** try another model from challenge 2 and see if the facts improve. + +You successfully completed challenge 4! 🚀🚀🚀 + +## Coach notes + +- **Always-retrieve versus agentic retrieval.** The two rows in the table are the discussion: always retrieving is simpler, a bit better here, and cannot skip the search for a machine status. Agentic retrieval lets one agent handle both kinds of question and choose, at the cost of one more model decision that can go wrong. For a single-purpose Q&A bot, always retrieve. For a copilot that also acts, let the agent decide. +- **Keep the document set small.** Embedding must finish in seconds on a CPU. +- **Stretch goal:** combine tools in one answer: retrieve the SOP, check the machine status and create a ticket in one turn. The reference code has a `multi_step=True` mode for this, where every step must call a tool and the model ends with a `final_answer` tool. **It is unreliable with `phi-4-mini`**: in the test it picked the wrong tool for a ticket request, dropped the key number from the answer, and once repeated one sentence until it ran out of tokens. Use it to show why the simple loop is the default, or to try a bigger model. +- The first call of every tool is slow (several seconds) because the model has to be loaded and warmed up. Warm the models up before the demo. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-05/solution-05.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-05/solution-05.md new file mode 100644 index 000000000..2ccf7d708 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-05/solution-05.md @@ -0,0 +1,130 @@ +# Walkthrough Challenge 5 - Train in the cloud, deploy at the edge + +[Previous Challenge Solution](../challenge-04/solution-04.md) - **[Home](../../Readme.md)** - [Next Challenge Solution](../challenge-06/solution-06.md) + +Duration: 60 minutes (shortcut: 25 minutes) + +## Prerequisites + +Please ensure that you successfully completed [Challenge 4](../../challenges/challenge-04.md) before continuing with this challenge. The full path also needs the Azure Machine Learning workspace from your lab page. + +### **Task 1 (full path): Train in Azure Machine Learning** + +1. Open [Azure Machine Learning studio](https://ml.azure.com) and select your workspace (`mlw-oic-...`). +2. **Compute** > **Compute instances** > **New**. Choose a small CPU size such as `Standard_DS3_v2`. Training takes a few seconds, so a small size is enough. +3. **Notebooks**: upload `resources/shortcut/train_fault_classifier.ipynb` and `resources/data/sensor_history.csv`, open the notebook and run all cells. You can also paste the code of [train_fault_classifier.py](../code/train_fault_classifier.py). +4. The notebook: + - loads the 2,400 labelled sensor readings, + - trains a random forest (about 94 percent accuracy on held-out data), + - exports it to **ONNX**, + - loads the ONNX file with ONNX Runtime and checks that it gives the same answers as the training library. +5. Download `fault_classifier.onnx` and `fault_labels.json` and copy them into your `work` folder, in a folder named `artifacts`. +6. **Stop** the compute instance (or delete it). It bills while it runs. + +💡 The point of the exercise is the **workflow**: cloud tools for training, a portable file for delivery, a small runtime on the device. This model is tiny, so you could also train it on your laptop with `python train_fault_classifier.py`. + +### **Task 1 (shortcut): Read the notebook and use the ready-made model** + +Read through `resources/shortcut/train_fault_classifier.ipynb` and copy `fault_classifier.onnx` and `fault_labels.json` from `resources/shortcut`. + +### **Task 2: Run the model on the device** + +[fault_tool.py](../code/fault_tool.py) loads the ONNX file with ONNX Runtime and **no** training library: + +```python +_session = ort.InferenceSession(str(ARTIFACTS / "fault_classifier.onnx"), providers=["CPUExecutionProvider"]) + +features = np.array([[machine[f] for f in _meta["features"]]], dtype=np.float32) +label_idx, probabilities = _session.run(None, {_session.get_inputs()[0].name: features}) +``` + +The file is 140 KB. A prediction takes a few milliseconds on the CPU: no GPU, no NPU and no network. + +### **Task 3: Make it a tool for the agent** + +`predict_fault(machine_id)` reads the machine's current readings (the same data as the status tool) and returns the predicted fault with its probability. Add its schema to the agent: + +```python +agent.run_agent(client, model, question, extra_tools=[(fault_tool.SCHEMA, fault_tool.predict_fault)]) +``` + +```powershell +python copilot.py "Predict the most likely fault for CNC-07 from its sensor readings." +``` + +```text + -> predict_fault({'machine_id': 'CNC-07'}) +The most likely fault for CNC-07 is bearing wear, with a confidence level of 98%. +``` + +Without help, the agent often does not make this call (see task 3b). The copilot in [copilot.py](../code/copilot.py) uses the router from task 3b, so this request works. + +### **Task 3b: Make the tool choice reliable** + +💥 **A small model picks the wrong tool surprisingly often.** With all four tools in front of it, `phi-4-mini` sent most fault requests to the SOP search instead of the classifier. We measured this with 23 requests that were written once and never used to tune anything ([routing_holdout.json](../../resources/data/eval/routing_holdout.json), run with [check_routing.py](../code/check_routing.py)): + +| `phi-4-mini` sees... | First tool correct | Fault prediction | +|----------------------|--------------------|------------------| +| all four tools (what we had) | 14 of 23 (61%) | 0 of 5 | +| the tool chosen by the router | **23 of 23** | 5 of 5 | + +The wording mattered a lot: *"Run the fault prediction model for CNC-07"* worked, while *"Predict the most likely fault for CNC-07 from its sensor readings"* or *"What does the classifier say is wrong with CMP-01?"* did not. The same prompts even routed differently from one session to the next, so there is no wording that "always works". + +**The fix is to shorten the list the model chooses from.** This is called *tool retrieval*: the same idea as the SOP search, applied to tools. Foundry Local's own documentation says `tool_choice="required"` is only best-effort, and research on tool retrieval reports large gains in selection accuracy when the model sees fewer, more relevant tools. [tool_router.py](../code/tool_router.py) does two things: + +1. **A tool whose required argument is missing is not a candidate.** Status, fault prediction and tickets all need a machine ID. A request that names none (*"At what vibration level must a spindle be stopped?"*) can only be a question for the SOPs. This one rule fixes most of the mix-ups. +2. **The rest is matched with the embedding model that is already loaded.** Every tool has a short *card*: what it is for and a few example requests. The request is embedded and compared with the cards. + +```python +router = ToolRouter(client, embed_model) +agent.run_agent(client, model, question, extra_tools=tools, router=router) # shortlist=1 by default +``` + +The agent then shows the model **only the best-matching tool**, and the model only fills in the arguments. The write tool is only shown if it is the best match, so the approval and the other protections from challenge 3 still apply. + +```text +Technician> Predict the most likely fault for CNC-07 from its sensor readings. + -> predict_fault({'machine_id': 'CNC-07'}) +The most likely fault for CNC-07 is bearing wear, with a confidence level of 98%. +``` + +Results with `phi-4-mini`, first tool correct: + +| Set | No router | Router, shows 1 tool | Router, shows 2 tools | +|-----|-----------|----------------------|-----------------------| +| Tuning set, 32 requests (used to build the cards) | 20 of 32 | **31 of 32** | 26 of 32 | +| Held-out set, 23 requests | 14 of 23 | **23 of 23** | not run | +| Held-out set on the **CPU variant** of the model | not run | **23 of 23** | not run | + +- **Showing two tools is worse than showing one.** With *status* and *fault prediction* side by side the model chose *status* for 5 of the 8 fault requests. So the router decides and the model does not. +- **The one miss** is *"CNC-07 shows error E-SPN-214. What should I do about it?"*: it names a machine and an error code, so both *status* and *SOP search* are reasonable. The model then wrote a procedure that is not from any SOP. A good discussion: when the router cannot tell, should the agent ask? +- **Be careful with these numbers.** The cards were written by us after we had seen the first failures, and the held-out set is only 23 requests. The held-out result shows the idea works. It does not prove that a new phrasing can never fail. Teams should add their own requests to the two JSON files. +- The router costs one embedding call per request (about 0.2 s on the laptop GPU). + + +### **Task 4: How far do you trust it?** + +| Machine | Reading | Predicted fault | Confidence | +|---------|---------|-----------------|------------| +| CNC-07 | vibration 9.4 mm/s, 78 °C | `bearing_wear` | 0.98 | +| CMP-01 | 104 °C, vibration 2.8 mm/s | `overheating` | 1.00 | +| CNC-03 | normal readings | `normal` | 0.99 | +| LATHE-05 | vibration 4.9 mm/s | `bearing_wear` | 0.67 | +| PRESS-02 | oil temperature warning E-HYD-031 | `low_lubrication` | 0.65 | + +- **CNC-07, CMP-01 and CNC-03:** high confidence, and they agree with the error codes and the SOPs. Trust them. +- **PRESS-02:** the classifier says low lubrication with 0.65 while the controller says *oil temperature high* (E-HYD-031). The error code and the SOP are facts from the machine. The classifier is advice. **A known error code beats a low-confidence prediction.** +- **LATHE-05:** 0.67 is an early warning at best. Worth a look at the next planned stop, not a ticket. + +A rule for the technician: a result below about 0.8 is a hint, not a diagnosis. The agent should show the confidence so the person can decide. + +You successfully completed challenge 5! 🚀🚀🚀 + +## Coach notes + +- **Decide the path per team at the start of the day**, based on experience and Azure access. Teams on the shortcut gain about 35 minutes. Give them a longer challenge 4 or the stretch goals. +- **Custom model serving in Foundry Local:** this MicroHack does **not** serve the classifier through Foundry Local. Foundry Local serves the catalog models, and a scikit-learn classifier is not a chat model. What we validated is the fallback from the outline: the ONNX classifier runs as an **agent tool next to** the small language model, with ONNX Runtime. A fine-tuned small language model served by Foundry Local is not covered here. +- **Cost:** the Azure Machine Learning workspace costs nothing while idle. The compute instance costs a few cents per hour. Remind teams to stop it. +- **Verified with the `az ml` command line** against the workspace that the lab automation creates: the compute instance (`Standard_DS3_v2`) was `Running` **3.7 minutes** after the request, and the training job (`pip install skl2onnx onnx`, then `train_fault_classifier.py`) ran in **2.5 minutes** on it. It reproduced the local result exactly: accuracy 0.940, the same confusion matrix, and an ONNX file that is **byte-identical** to `resources/shortcut/fault_classifier.onnx`. So the shortcut file is a faithful stand-in for a team's own result, and a team's file can be checked against it with a file hash. +- **Not verified:** the clicks in the Azure Machine Learning **studio** (notebooks, kernel start-up) were not exercised, only the same steps through the command line. Plan about 4 minutes for the compute instance to start, and do the studio steps once before the first event. +- **Stretch:** compare the classifier's verdict with what the language model says about the same readings (*"given vibration 9.4 mm/s and 78 °C, what is wrong?"*) and discuss which to trust when. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-06/solution-06.md b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-06/solution-06.md new file mode 100644 index 000000000..4e7143820 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/challenge-06/solution-06.md @@ -0,0 +1,193 @@ +# Walkthrough Challenge 6 - Cut the cord + +[Previous Challenge Solution](../challenge-05/solution-05.md) - **[Home](../../Readme.md)** - [Finish](../../challenges/finish.md) + +Duration: 30 minutes + +## Prerequisites + +Please ensure that you successfully completed [Challenge 5](../../challenges/challenge-05.md) before continuing with this challenge. The reference copilot is [copilot.py](../code/copilot.py): one agent with four tools (machine status, tickets, fault prediction, SOP search). + +### **Task 1: Run it while connected and write down the timings** + +```powershell +python copilot.py "What is the status of machine CNC-07?" "Predict the most likely fault for CNC-07 from its sensor readings." "At what vibration level must a CNC spindle be stopped?" "Open a high priority ticket for CNC-07: spindle vibration above the stop limit." +``` + +Reference timings on the laptop (Snapdragon X Elite, GPU variant), **connected**, for the first run after the service started. Your numbers will differ with your hardware. + +| Step | Laptop | +|------|--------| +| Machine status (tool call and answer) | 13 s | +| Fault prediction (ONNX model and answer) | 9 s | +| SOP question (split, embed, search, answer) | 17 s | +| Ticket (tool call and confirmation) | 6 s | + +The ticket time includes the seconds the technician needs to type `y`. The first request is the slowest because the models are still cold. The times for the Windows VM are in task 5. + +### **Task 2: Get ready, then disconnect** + +💥 **Start everything before you disconnect, and do not restart the `foundry` service while you are offline.** The service loads its model catalog from the internet when it starts. A service that is already running, with its models loaded, keeps answering offline. A service that is restarted offline cannot find any model, even a model that is on the disk (see the coach notes). + +1. Make sure the service is running and both models are loaded: + + ```powershell + foundry server status + foundry model list --loaded + ``` + +2. Ask one question of each kind, so every model is warm and the SOP index exists. + +💡 **If your device must be able to restart while it is offline**, start the models with [serve_offline.py](../code/serve_offline.py) instead of the `foundry` service. It uses the Foundry Local Python SDK, which finds the models in your local cache without contacting the internet: + +```powershell +pip install foundry-local-sdk # while you are still online +python serve_offline.py # serves phi-4-mini and the embedding model +$env:FOUNDRY_LOCAL_URL = "http://127.0.0.1:5272/v1" # in the terminal where you run the copilot +``` + +By default it uses the **CPU** variants of the models. On the VM that is what you get anyway. On a laptop with a GPU or NPU, add `--accelerate` to use those variants instead. That works offline as long as you ran a GPU model once while you were online (see the coach notes). + +### **Task 3: Disconnect** + +**Laptop**, in a terminal started as administrator. This blocks outbound traffic to the internet and leaves the loopback address that Foundry Local uses alone: + +```powershell +New-NetFirewallRule -DisplayName "Offline copilot test" -Direction Outbound -Action Block -RemoteAddress Internet +``` + +Remove it afterwards with `Remove-NetFirewallRule -DisplayName "Offline copilot test"`. Turning Wi-Fi off or unplugging the cable works as well. + +**VM**: add an outbound rule to the network security group of the VM. It blocks the internet and keeps your Remote Desktop session and the Azure management path: + +```powershell +az network nsg rule create -g --nsg-name -n deny-internet-out ` + --priority 100 --direction Outbound --access Deny --protocol '*' ` + --source-address-prefixes '*' --source-port-ranges '*' ` + --destination-address-prefixes Internet --destination-port-ranges '*' +``` + +Wait about 30 seconds for the rule to take effect. + +### **Task 4: Check that you are really offline** + +[offline_check.py](../code/offline_check.py) checks three things: the model service is on a loopback address, the internet is **not** reachable, and a local answer still comes back. + +```powershell +python offline_check.py +``` + +Expected output when the block works (this is the real output from the VM test below): + +```text +Model service : http://127.0.0.1:59668/v1 +Runs locally : yes +Internet : not reachable (offline) +Local answers : [2.48, 1.05] s +``` + +If it says `REACHABLE`, the block does not work yet. Do not continue until it says `not reachable`. + +### **Task 5: Run the whole scenario again** + +Repeat the questions from task 1 and write down the timings. Every capability from challenges 3 to 5 must work: the tools, the SOP search with sources, the ONNX model and the ticket. + +Offline results on the VM (8 vCPU, no GPU, CPU variants), with the internet blocked at the network level **and the service freshly started while offline** with `serve_offline.py`: + +| Step | Offline, on the VM | +|------|--------------------| +| Machine status (first request after the start, models still cold) | 26 s | +| Fault prediction (ONNX model and answer) | 19 s | +| SOP question (split, embed, search, answer) | 38 s | +| Ticket (tool call and confirmation) | 13.5 s | +| Error code plus machine ID (goes to the SOP search) | 40 s | +| The very first chat call after the start | 26 s, the next one 0.4 s | + +In an earlier run with the `foundry` service (started online, block applied afterwards) the connected and the offline times were the same within normal variation: status 18.5 s against 20.1 s, SOP question 25.7 s against 26.4 s. **Nothing in the question path needs the internet.** + +### **Task 6: What still needs the cloud?** + +Go through the copilot and the Foundry Local service and ask *"does this need the internet?"* A good list: + +| Needs the internet | When | +|--------------------|------| +| Starting or restarting the Foundry Local service | Whenever it starts (it fetches the model catalog) | +| Downloading a model or a model update | Once, and when a model is refreshed | +| Updating Foundry Local itself and the Python packages | On a schedule, from a controlled place | +| Syncing the maintenance tickets to the central system | When the device is back online | +| Retraining the fault classifier with new sensor history | Periodically, in the cloud (challenge 5) | +| Monitoring and logs to a central place | When connected | + +💡 A good routine for the stretch goal: when the device reconnects, it uploads the new tickets, then checks for a newer model, downloads it, tests it with the evaluation set from challenge 4, and only then switches to it. + +You successfully completed challenge 6! 🚀🚀🚀 + +## Coach notes + +### What was verified (on a Windows 11 VM in Azure, 8 vCPU, no GPU) + +- The outbound deny rule above blocks the internet (`aka.ms`, `huggingface.co` and `pypi.org` all time out) and the **Azure management path keeps working**: a coach can still run commands on the VM with `az vm run-command`. Do not add an inbound deny rule. +- With the service started and the models loaded **before** the block, the whole copilot works offline, with the same timings as connected. +- Starting the service takes 2 seconds, loading `phi-4-mini` and the embedding model about 17 seconds, and the first chat answer after that 0.6 s. + +### What breaks offline: restarting the service + +After `foundry server restart` with the internet blocked, `foundry model load` and every other command that names a model failed with `No model matching 'phi-4-mini' was found in the catalog`, **also for a model that was already downloaded** and also when the full variant ID was used. The service log shows why: + +```text +GetModelVariant: 'Phi-4-mini-instruct-generic-cpu:5' not in cache, fetching from catalog source. +Region detection probe failed (status 0); defaulting to 'centralus'. +RegionFallback: region 'centralus' unhealthy (transport failure); trying next candidate. +... (7 regions, about 21 seconds each) +``` + +It tries 7 regions in turn, which is about 2.5 minutes of waiting, and then gives up. Teams that block the network and then restart the service (or reboot the VM) will lose the copilot and will think the network block broke something. + +- **Tell teams in the briefing:** start and warm up first, disconnect second, do not restart the `foundry` service. + +### The fix we verified: start the models from the SDK + +[serve_offline.py](../code/serve_offline.py) starts the models with the Foundry Local Python SDK. It gives the SDK a catalog address that fails at once (`http://127.0.0.1:9`, nothing listens there), so the SDK never goes looking for the 7 cloud regions. It finds the models in the local cache on the disk and serves them on an OpenAI-compatible address. [fl.py](../code/fl.py) reads what it serves when `FOUNDRY_LOCAL_URL` is set. + +Verified on a fresh Windows VM (SDK installed with `pip` and the two CPU model variants downloaded with the SDK while online, 93 s for 5.8 GB): + +- Started online: ready in 30 s. Then the internet was blocked and the launcher was **stopped and started again while offline**: ready again in 30 s, both models served. The `foundry` service could not do this. +- Offline, against that freshly started service: all 5 copilot requests went to the right tool, and the 23 held-out routing requests scored 23 of 23 (see [solution 5](../challenge-05/solution-05.md)). +- It runs fine as a Windows scheduled task under the `SYSTEM` account, so it can start at boot without anyone signing in. + +Details and limits: + +- **GPU and NPU variants (`--accelerate`). Verified on the laptop with the network blocked for the process** (a firewall rule for the Python interpreter, administrator terminal). The root cause of the problem we first saw: + - The **NPU** (QNN) execution provider is installed with Foundry Local, registers offline in about 2 s, and the NPU model loaded. + - The **GPU** (WebGPU) execution provider is downloaded the first time a GPU model runs online and kept in `~/.foundry/ep/webgpu-ep`. But the SDK **tries to download it again at every start** (`https://foundrypackages-....azurefd.net/webgpu_ep_0.2.1_win-arm64.zip`). Offline that fails, the provider is not registered, and every GPU model refuses to load with `requires WebGpuExecutionProvider which is not registered`. The call to register the providers still reports `success=True`, which is misleading. + - The native library has an undocumented environment variable, **`FOUNDRY_LOCAL_WEBGPU_EP_LIBRARY`**, that points to a provider on the disk. With it set to the copy in `~/.foundry/ep`, both providers register offline in 1.6 s with no download, the GPU variant of `phi-4-mini` loads and answers in 1.3 s. `serve_offline.py --accelerate` sets it for you. + - With the launcher under the same block: CPU variants ready in 18 s, GPU variants ready in 12 s, and the copilot answered correctly in both (fault prediction 8.4 s on CPU against 5.7 s on GPU). + - **Caveats.** The variable is undocumented, so it may change in a later version (Foundry Local 0.11.0, SDK 2.1.0 here). The provider must have been downloaded once, so run a GPU model online before you disconnect. We cut the network for one process and did not reboot the laptop, but everything the launcher needs is a file on the disk. A CUDA provider has a similar variable (`FOUNDRY_LOCAL_CUDA_EP_LIBRARY`), which we did not test. +- The `foundry` command line cannot see a service that the SDK started (`foundry model list --loaded` will not list its models). That is why `fl.py` reads a small file. +- The first chat request after any start is slow on a CPU (26 s here for the first one, 0.4 s for the next). Send a warm-up request before the technician asks. +- On the VM, embedding the 25 SOP chunks takes 36 s on the CPU and one SOP retrieval about 1 s. The ONNX prediction takes about 1 ms. + +### Other observations + +- **The service keeps trying to reach the internet in the background.** Even in a steady state, `Get-NetTCPConnection` shows connections in the `SynSent` state to Azure addresses on port 443 (for example `20.42.73.27`) that never complete. They do not slow the answers. This is useful for the discussion about *"does it phone home?"*: with the network blocked the attempts fail quietly and nothing in the copilot depends on them. We did not find out what the service sends. +- **Python packages must be installed before the disconnect.** `pip` has no internet either. +- **The block must not stop the loopback address.** `127.0.0.1` is not "Internet" for the firewall rule above. If a team blocks "all" outbound traffic and the copilot stops, this is why. + +### Verified on a laptop (Windows 11 ARM64) + +`New-NetFirewallRule -Direction Outbound -Action Block -RemoteAddress Internet` was run in an administrator terminal. With the rule active, `offline_check.py` printed `not reachable (offline)`, the local model still answered (1.5 s, loopback is not "Internet") and a full copilot question worked. We scoped the rule to the Foundry service and to Python with `-Program`, because the documented rule without `-Program` would have cut our own session. **The rule exactly as written in task 3 (for all programs) was not run.** Two things to know: + +- If you scope the rule to Python with `-Program`, use the **real interpreter**, not the one in the virtual environment. `.venv\Scripts\python.exe` is only a launcher that starts the real one, so a rule for it blocks nothing (our first attempt printed `REACHABLE`). `python -c "import sys; print(sys._base_executable)"` shows the real path. The rule in task 3 has no `-Program`, so it does not have this problem. +- Remove the rule when you are done, also if a test fails halfway. A `try ... finally` around the test is a good habit. + +### A Foundry Local documentation claim that does not hold for the CLI service + +The documentation says: *"After the initial download, models run entirely offline from the local cache."* That is true for a service that is already running, and for the SDK as shown above, but **not** for a `foundry` service that is restarted offline. + +### Not verified + +- Turning Wi-Fi off or unplugging the cable. +- A real reboot of a GPU or NPU device with no network. We blocked the network for the launcher process and did not restart the machine. +- The `foundry` service itself with a local provider: we only fixed the SDK path. + +**Stretch:** design the reconnection routine from task 6 and write the ticket-sync part. diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/agent.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/agent.py new file mode 100644 index 000000000..84ac547e5 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/agent.py @@ -0,0 +1,152 @@ +"""Challenges 3 and 4: a local agent that calls tools. + +run_agent(...) simple loop: one forced tool call, then the model answers (challenge 3) +run_agent(..., multi_step=True) the model keeps calling tools until it calls final_answer (challenge 4) +""" +import json +import re +import sys + +import fl +from tools import IMPLEMENTATIONS, TOOL_SCHEMAS + +SYSTEM = ( + "You are a maintenance assistant for technicians at Contoso Precision Machining. " + "Machine data and tickets are only available through your tools: never guess a machine reading. " + "For procedures, limits, torque values, intervals and error codes, call search_sops and answer only from the " + "excerpts. Keep every number and unit from the excerpts. Cite only SOP IDs that appear in the excerpts. " + "When you call search_sops, copy the technician's whole question into it, word for word, so no topic is lost. " + "If the excerpts do not contain the answer, say you could not find it in the SOPs. " + "After you receive tool results, answer in at most three short sentences and then stop." +) +MULTI_STEP_SYSTEM = SYSTEM + " When you have everything you need, call final_answer." +FINAL_ANSWER = {"type": "function", "function": { + "name": "final_answer", + "description": "Give the technician the final answer once you have all the information you need.", + "parameters": {"type": "object", + "properties": {"answer": {"type": "string", "description": "The answer. Keep all numbers and units."}}, + "required": ["answer"]}}} + +WRITE_TOOLS = {"create_maintenance_ticket"} +WRITE_INTENT = re.compile(r"\b(?:open|create|log|raise|file|make)\b[^.?!]{0,40}\bticket\b", re.I) +KNOWN_SOPS = {p.name.split("_")[0] for p in (fl.DATA / "sops").glob("*.md")} + + +def ask_technician(name, args): + return input(f" The assistant wants to run {name}({args}). Allow? [y/N] ").strip().lower() == "y" + + +SOP_REFERENCE = re.compile(r"SOP(?:\s+ID)?[\s:-]+[A-Z0-9][A-Z0-9-]*") +# "A ticket has been opened", "ticket MT-1001", "I created a ticket" - claims in the past tense, not SOP advice like "create a ticket". +TICKET_CLAIM = re.compile( + r"ticket[^.]{0,60}\b(?:has|have|had|was|is)\s+(?:now\s+|successfully\s+)?(?:been\s+)?(?:opened|created|logged|raised)\b" + r"|\b(?:I|we)(?:\s+have|'ve)?\s+(?:opened|created|logged|raised)\b[^.]{0,30}ticket" + r"|\bMT-\d{3,}\b", re.I) + + +def scrub_citations(text): + """A small model can invent a source (for example 'SOP ID 1024'). Drop every sentence that cites an SOP that does not exist.""" + sentences = re.split(r"(?<=[.!?])\s+", text.strip()) + return " ".join(s for s in sentences if all(ref in KNOWN_SOPS for ref in SOP_REFERENCE.findall(s))) + + +def first_tool_call(reply): + """Return (id, name, arguments) of the first tool call, or None. + + Some models write the call as text (for example <|tool_call|>[{...}]) instead of a structured call, + so we parse that form too. + """ + if reply.tool_calls: + call = reply.tool_calls[0] + return call.id, call.function.name, json.loads(call.function.arguments or "{}") + match = re.search(r"<\|?tool_call\|?>\s*(\[.*\]|\{.*\})", reply.content or "", re.S) + if match: + try: + data = json.loads(match.group(1)) + data = data[0] if isinstance(data, list) else data + return "call_from_text", data["name"], data.get("arguments") or data.get("parameters") or {} + except (ValueError, KeyError, IndexError): + return None + return None + + +def run_agent(client, model, question, extra_tools=None, approve=ask_technician, multi_step=False, + router=None, shortlist=1, max_steps=6, verbose=True): + """extra_tools is a list of (schema, function) pairs added to the two built-in tools. + + router (a ToolRouter) shortlists the tools shown to the model for this request. shortlist is how many. + + Tools that change something (WRITE_TOOLS) only run if approve() says yes. A small model can + decide to create a ticket nobody asked for, so a person stays in the loop. + """ + schemas = list(TOOL_SCHEMAS) + [s for s, _ in (extra_tools or [])] + ([FINAL_ANSWER] if multi_step else []) + functions = dict(IMPLEMENTATIONS) | {s["function"]["name"]: f for s, f in (extra_tools or [])} + # A tool that changes something is only offered when the technician asked for it. A model cannot invent a + # ticket it is never offered, and a ticket request cannot be sent to the wrong tool. + wants_write = bool(WRITE_INTENT.search(question)) + read_only = [s for s in schemas if s["function"]["name"] not in WRITE_TOOLS] + write_only = [s for s in schemas if s["function"]["name"] in WRITE_TOOLS] + first_step = write_only if wants_write else read_only + if router is not None: + # Tool shortlisting: show the model only the best-matching tools. A write tool is only shown if it is the best match. + best = router.candidates(question)[:shortlist] + if best and best[0] in WRITE_TOOLS: + wants_write, first_step = True, [s for s in schemas if s["function"]["name"] == best[0]] + else: + first_step = [s for s in read_only if s["function"]["name"] in best] or read_only + messages = [{"role": "system", "content": MULTI_STEP_SYSTEM if multi_step else SYSTEM}, + {"role": "user", "content": question}] + sources = [] + + def finish(text): + text = scrub_citations(text or "") + # No write tool ran in this branch, so any claim that a ticket was opened is made up. + if TICKET_CLAIM.search(text): + return "No ticket was created: the assistant did not call the ticket tool. Please ask again, for example: 'Open a high priority ticket for CNC-07: '." + # Small models forget to cite, so the app attaches the SOPs it really retrieved. + if sources and "not find" not in text.lower(): + text += "\n Sources: " + "; ".join(sources) + return text + + for step in range(max_steps): + # Small models often describe a tool call instead of making one, so a call is mandatory. + # In the simple loop only the first step is forced and the next step must answer in text. + choice = "required" if (multi_step or step == 0) else "none" + offered = first_step if step == 0 else (schemas if wants_write else read_only) + reply = client.chat.completions.create( + model=model, messages=messages, tools=offered, tool_choice=choice, temperature=0, max_tokens=500, + ).choices[0].message + call = first_tool_call(reply) + if call is None: + return finish(reply.content) + call_id, name, args = call + if name == "final_answer": + return finish(args.get("answer", "")) + if name not in functions: + return finish(f"(the model asked for an unknown tool: {name})") + if verbose: + print(f" -> {name}({args})") + # One tool call at a time keeps the history valid and stops a model from inventing extra actions. + messages.append({"role": "assistant", "content": None, "tool_calls": [ + {"id": call_id, "type": "function", "function": {"name": name, "arguments": json.dumps(args)}}]}) + if name in WRITE_TOOLS: + # The app, not the model, reports what happened. A small model can claim a ticket exists when it never called the tool. + if not approve(name, args): + return "Cancelled: the technician did not approve, so nothing was changed." + result = functions[name](**args) + if "error" in result: + return f"Nothing was changed: {result['error']}" + return (f"Ticket {result['ticket_id']} created for {result['machine_id']} " + f"({result['priority']} priority): {result['summary']}") + result = functions[name](**args) + if isinstance(result, dict) and isinstance(result.get("excerpts"), list): + sources += [s for s in (f"{e['sop']} rev {e['revision']} / {e['section']}" for e in result["excerpts"]) + if s not in sources] + messages.append({"role": "tool", "tool_call_id": call_id, "content": json.dumps(result)}) + return "(stopped: too many steps)" + + +if __name__ == "__main__": + model = fl.load(sys.argv[2] if len(sys.argv) > 2 else "phi-4-mini") + question = sys.argv[1] if len(sys.argv) > 1 else "What is the status of machine CNC-07?" + print(run_agent(fl.client(), model, question)) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/benchmark.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/benchmark.py new file mode 100644 index 000000000..5f47c65e8 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/benchmark.py @@ -0,0 +1,100 @@ +"""Challenge 2: compare edge models on speed, memory, tool calling and answer quality.""" +import json +import re +import sys +import time + +import fl +from tools import TOOL_SCHEMAS + +SYSTEM = ("You are a maintenance assistant for technicians. Machine data is only available through your tools. " + "Never guess a machine reading.") + + +def strip_thinking(text): + return re.sub(r".*?", "", text or "", flags=re.S).strip() + + +def speed(client, model): + start = time.perf_counter() + first = None + chunks = 0 + stream = client.chat.completions.create( + model=model, stream=True, temperature=0, max_tokens=120, + messages=[{"role": "user", "content": "Explain in three sentences why bearings fail."}]) + for part in stream: + # Reasoning models stream their thinking in a separate field. It costs time, so we count it. + delta = part.choices[0].delta if part.choices else None + if delta and (delta.content or getattr(delta, "reasoning_content", None)): + chunks += 1 + first = first or time.perf_counter() + end = time.perf_counter() + return round((first or end) - start, 2), round(chunks / max(end - (first or start), 0.001), 1) + + +def tool_calling(client, model, choice): + cases = json.loads((fl.DATA / "eval" / "tool_eval.json").read_text(encoding="utf-8")) + passed = 0 + for case in cases: + try: + reply = client.chat.completions.create( + model=model, temperature=0, max_tokens=150, tools=TOOL_SCHEMAS, tool_choice=choice, + messages=[{"role": "system", "content": SYSTEM}, {"role": "user", "content": case["prompt"]}], + ).choices[0].message + except Exception: + continue # a failed request counts as a failed case + calls = reply.tool_calls or [] + if case["expected_tool"] is None: + ok = not calls + else: + # Exactly one call: a model that also invents a second call (for example a ticket) fails. + ok = len(calls) == 1 and calls[0].function.name == case["expected_tool"] + if ok: + args = json.loads(calls[0].function.arguments or "{}") + ok = all(str(args.get(k, "")).lower() == v.lower() for k, v in case["expected_args"].items()) + passed += ok + return f"{passed}/{len(cases)}" + + +def quality(client, model): + """Give the model the right SOP and check that the answer contains the expected facts.""" + cases = [c for c in json.loads((fl.DATA / "eval" / "retrieval_eval.json").read_text(encoding="utf-8")) + if c["expected_sop"]] + sops = {p.name.split("_")[0]: p.read_text(encoding="utf-8") for p in (fl.DATA / "sops").glob("*.md")} + passed = 0 + for case in cases: + reply = client.chat.completions.create( + model=model, temperature=0, max_tokens=200, + messages=[{"role": "system", "content": "Answer briefly using only this SOP.\n\n" + sops[case["expected_sop"]]}, + {"role": "user", "content": case["question"]}], + ).choices[0].message.content + answer = strip_thinking(reply).lower() + passed += all(f.lower() in answer for f in case["must_include"]) + return f"{passed}/{len(cases)}" + + +if __name__ == "__main__": + aliases = sys.argv[1:] or ["phi-4-mini", "qwen2.5-1.5b", "qwen3-1.7b"] + client = fl.client() + rows = [] + for alias in aliases: + print(f"Benchmarking {alias} ...", flush=True) + t = time.perf_counter() + model = fl.load(alias) + load_s = round(time.perf_counter() - t, 1) + info = fl.loaded_models()[alias] + ttft, tps = speed(client, model) + rows.append({ + "model": alias, "device": info["device"], "size_mb": info["fileSizeMb"], "load_s": load_s, + "first_token_s": ttft, "tokens_per_s": tps, + "tools_auto": tool_calling(client, model, "auto"), + "tools_required": tool_calling(client, model, "required"), + "quality": quality(client, model), + }) + _ = fl._foundry("model", "unload", alias, as_json=False) + cols = list(rows[0]) + print("\n" + " | ".join(cols)) + for r in rows: + print(" | ".join(str(r[c]) for c in cols)) + with open("benchmark_results.json", "w", encoding="utf-8") as f: + json.dump(rows, f, indent=2) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/check_routing.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/check_routing.py new file mode 100644 index 000000000..670b9b4a2 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/check_routing.py @@ -0,0 +1,69 @@ +"""Challenge 4: how often does the agent pick the right tool first? + + python check_routing.py tuning set, tool shortlist of 1 + python check_routing.py holdout requests the router was not tuned on + python check_routing.py tuning baseline no router: the model sees all tools + python check_routing.py tuning top2 router shortlists 2 tools, the model chooses + +Set ROUTING_MODEL_ID to a full model ID (for example Phi-4-mini-instruct-generic-cpu:5) to test one hardware variant. +Only the first tool the model picks is counted. Tickets are never created (the approval is always 'no'). +""" +import json +import os +import sys +import time + +import agent +import fault_tool +import fl +import retrieval +from tool_router import ToolRouter + +SETS = {"tuning": "routing_eval.json", "holdout": "routing_holdout.json"} +first_tool = [] # the tool the model picked first for the current request +_original = agent.first_tool_call + + +def _spy(reply): + call = _original(reply) + if call and not first_tool: + first_tool.append(call[1]) + return call + + +agent.first_tool_call = _spy + +if __name__ == "__main__": + dataset = sys.argv[1] if len(sys.argv) > 1 else "tuning" + strategy = sys.argv[2] if len(sys.argv) > 2 else "top1" + cases = json.loads((fl.DATA / "eval" / SETS[dataset]).read_text(encoding="utf-8")) + + client = fl.client() + embed_model = fl.load(retrieval.EMBED_MODEL) + chat_model = os.environ.get("ROUTING_MODEL_ID") or fl.load(retrieval.CHAT_MODEL) + chunks, vectors = retrieval.load_index() if retrieval.INDEX.exists() else retrieval.build_index(client, embed_model) + tools = [(fault_tool.SCHEMA, fault_tool.predict_fault), + retrieval.search_tool(client, chat_model, embed_model, chunks, vectors)] + router = None if strategy == "baseline" else ToolRouter(client, embed_model) + shortlist = 2 if strategy == "top2" else 1 + + right, misses, start = 0, [], time.perf_counter() + per_tool = {} + for case in cases: + first_tool.clear() + agent.run_agent(client, chat_model, case["request"], extra_tools=tools, approve=lambda *_: False, + router=router, shortlist=shortlist, verbose=False) + picked = first_tool[0] if first_tool else "(no tool)" + ok = picked == case["expected_tool"] + right += ok + stats = per_tool.setdefault(case["expected_tool"], [0, 0]) + stats[0] += ok + stats[1] += 1 + if not ok: + misses.append((case["id"], case["request"][:58], case["expected_tool"], picked)) + print(f"[{dataset} / {strategy} / {chat_model}]") + for tool, (ok, n) in per_tool.items(): + print(f" {ok}/{n} {tool}") + print(f"first tool correct: {right}/{len(cases)} ({time.perf_counter() - start:.0f} s)") + for m in misses: + print(" miss:", m) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/compare_cloud.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/compare_cloud.py new file mode 100644 index 000000000..26c955704 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/compare_cloud.py @@ -0,0 +1,42 @@ +"""Challenge 6 (optional): compare local latency with a cloud model. + +Set these environment variables first (any OpenAI-compatible cloud endpoint works): + CLOUD_BASE_URL for example https://.openai.azure.com/openai/v1 + CLOUD_API_KEY the key for that endpoint + CLOUD_MODEL the deployment name, for example gpt-4.1-mini +""" +import os +import statistics +import sys +import time + +from openai import OpenAI + +import fl + +QUESTIONS = [ + "In one sentence, what causes spindle bearing wear?", + "In one sentence, why must a hydraulic press be depressurised before maintenance?", + "In one sentence, what is lockout/tagout?", +] + + +def time_calls(client, model, runs=3): + samples = [] + for _ in range(runs): + for q in QUESTIONS: + start = time.perf_counter() + client.chat.completions.create(model=model, max_tokens=60, messages=[{"role": "user", "content": q}]) + samples.append(time.perf_counter() - start) + return statistics.median(samples), max(samples) + + +if __name__ == "__main__": + local_median, local_max = time_calls(fl.client(), fl.load("phi-4-mini")) + print(f"local median {local_median:.2f} s worst {local_max:.2f} s") + missing = [v for v in ("CLOUD_BASE_URL", "CLOUD_API_KEY", "CLOUD_MODEL") if not os.environ.get(v)] + if missing: + sys.exit(f"Cloud comparison skipped. Set: {', '.join(missing)}") + cloud = OpenAI(base_url=os.environ["CLOUD_BASE_URL"], api_key=os.environ["CLOUD_API_KEY"]) + cloud_median, cloud_max = time_calls(cloud, os.environ["CLOUD_MODEL"]) + print(f"cloud median {cloud_median:.2f} s worst {cloud_max:.2f} s") diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/copilot.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/copilot.py new file mode 100644 index 000000000..9760a621a --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/copilot.py @@ -0,0 +1,49 @@ +"""Challenge 6: the complete offline maintenance copilot. + +One agent with four tools: machine status, maintenance tickets, fault prediction (ONNX) and SOP search. +A ToolRouter shortlists the tool for each request, because a small model picks the wrong tool too often when it sees all four. +""" +import sys +import time + +import agent +import fault_tool +import fl +import retrieval +from tool_router import ToolRouter + +CHAT_MODEL = "phi-4-mini" + + +def build(): + client = fl.client() + chat_model = fl.load(CHAT_MODEL) + embed_model = fl.load(retrieval.EMBED_MODEL) + chunks, vectors = retrieval.load_index() if retrieval.INDEX.exists() else retrieval.build_index(client, embed_model) + tools = [(fault_tool.SCHEMA, fault_tool.predict_fault), + retrieval.search_tool(client, chat_model, embed_model, chunks, vectors)] + return client, chat_model, tools, ToolRouter(client, embed_model) + + +def main(): + client, chat_model, tools, router = build() + print("Offline maintenance copilot. Type a question, or 'exit'.") + queue = sys.argv[1:] # questions given on the command line run once, without a prompt + scripted = bool(queue) + while True: + if scripted: + if not queue: + return + question = queue.pop(0) + print(f"\nTechnician> {question}") + else: + question = input("\nTechnician> ").strip() + if question.lower() in ("", "exit", "quit"): + return + start = time.perf_counter() + print(agent.run_agent(client, chat_model, question, extra_tools=tools, router=router)) + print(f" ({time.perf_counter() - start:.1f} s)") + + +if __name__ == "__main__": + main() diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/evaluate_compound.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/evaluate_compound.py new file mode 100644 index 000000000..fcf64af5a --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/evaluate_compound.py @@ -0,0 +1,38 @@ +"""Challenge 4: does the agent retrieve more than once when a question needs it?""" +import json + +import agent +import fl +import retrieval as r + + +def evaluate(repeats=2): + client = fl.client() + embed_model, chat_model = fl.load(r.EMBED_MODEL), fl.load(r.CHAT_MODEL) + chunks, vectors = r.build_index(client, embed_model) + schema, search = r.search_tool(client, chat_model, embed_model, chunks, vectors) + calls = [] + + def counting_search(query): + result = search(query) + calls.extend(result["queries"]) + return result + + cases = json.loads((fl.DATA / "eval" / "compound_eval.json").read_text(encoding="utf-8")) + multi = both = facts = runs = 0 + for case in cases: + for _ in range(repeats): + calls.clear() + text = agent.run_agent(client, chat_model, case["question"], extra_tools=[(schema, counting_search)], + approve=lambda *_: False, verbose=False) + runs += 1 + multi += len(calls) >= 2 + both += all(s in text for s in case["expected_sops"]) + facts += all(f.lower() in text.lower() for f in case["must_include"]) + print(f"{case['id']} topics={len(calls)} both_sops={all(s in text for s in case['expected_sops'])} " + f"facts={all(f.lower() in text.lower() for f in case['must_include'])}") + print(f"\nretrieved more than once {multi}/{runs} | both SOPs found {both}/{runs} | all facts {facts}/{runs}") + + +if __name__ == "__main__": + evaluate() diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/evaluate_retrieval.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/evaluate_retrieval.py new file mode 100644 index 000000000..9b3a7f878 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/evaluate_retrieval.py @@ -0,0 +1,54 @@ +"""Challenge 4: measure retrieval and answer quality against the evaluation set. + + python evaluate_retrieval.py fixed pipeline: always retrieve, then answer + python evaluate_retrieval.py agent the agent decides whether to call search_sops +""" +import json +import sys + +import agent +import fl +import retrieval as r + + +REFUSAL_PHRASES = ("could not find", "not find", "not specified", "not available", "not mentioned", + "no information", "not contain", "not in the") + + +def is_refusal(text): + return any(p in text.lower() for p in REFUSAL_PHRASES) + + +def evaluate(mode="pipeline"): + client = fl.client() + embed_model, chat_model = fl.load(r.EMBED_MODEL), fl.load(r.CHAT_MODEL) + chunks, vectors = r.build_index(client, embed_model) + search = r.search_tool(client, chat_model, embed_model, chunks, vectors) + cases = json.loads((fl.DATA / "eval" / "retrieval_eval.json").read_text(encoding="utf-8")) + hit = correct = cited = refused = 0 + for case in cases: + if mode == "agent": + text = agent.run_agent(client, chat_model, case["question"], extra_tools=[search], + approve=lambda *_: False, verbose=False) + sources = [(case["expected_sop"],)] if case["expected_sop"] and case["expected_sop"] in text else [] + else: + text, sources = r.answer(client, chat_model, embed_model, chunks, vectors, case["question"]) + if not case["expected_sop"]: + ok = is_refusal(text) + refused += ok + print(f"{case['id']} {'PASS' if ok else 'FAIL'} (should refuse) -> {text[:70]!r}") + continue + facts = all(f.lower() in text.lower() for f in case["must_include"]) + cites = case["expected_sop"] in text + correct += facts + cited += cites + hit += any(s[0] == case["expected_sop"] for s in sources) if mode == "pipeline" else cites + print(f"{case['id']} facts={facts} cited={cites}") + n = sum(1 for c in cases if c["expected_sop"]) + label = "retrieval hit" if mode == "pipeline" else "right SOP named" + print(f"\n[{mode}] {label} {hit}/{n} | correct facts {correct}/{n} | cites right SOP {cited}/{n} | " + f"refused unanswerable {refused}/{len(cases) - n}") + + +if __name__ == "__main__": + evaluate(sys.argv[1] if len(sys.argv) > 1 else "pipeline") diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/fault_tool.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/fault_tool.py new file mode 100644 index 000000000..5768815b6 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/fault_tool.py @@ -0,0 +1,40 @@ +"""Challenge 5: the cloud-trained ONNX fault classifier, exposed as an agent tool.""" +import json +from pathlib import Path + +import numpy as np +import onnxruntime as ort + +import fl +from tools import get_machine_status + +# Use your own trained model if you made one (challenge 5, full path), otherwise the shortcut artifact. +_own = Path(__file__).resolve().parent / "artifacts" +ARTIFACTS = _own if (_own / "fault_classifier.onnx").exists() else fl.DATA.parent / "shortcut" +_meta = json.loads((ARTIFACTS / "fault_labels.json").read_text(encoding="utf-8")) +# CPU only: the model is tiny, so it runs in milliseconds with no GPU, no NPU and no network. +_session = ort.InferenceSession(str(ARTIFACTS / "fault_classifier.onnx"), providers=["CPUExecutionProvider"]) + + +def predict_fault(machine_id: str) -> dict: + machine = get_machine_status(machine_id) + if "error" in machine: + return machine + features = np.array([[machine[f] for f in _meta["features"]]], dtype=np.float32) + label_idx, probabilities = _session.run(None, {_session.get_inputs()[0].name: features}) + ranked = sorted(zip(_meta["labels"], probabilities[0].tolist()), key=lambda x: -x[1]) + return { + "machine_id": machine["id"], + "predicted_fault": _meta["labels"][int(label_idx[0])], + "confidence": round(ranked[0][1], 2), + "other_candidates": [{"fault": n, "probability": round(p, 2)} for n, p in ranked[1:3] if p >= 0.05], + } + + +SCHEMA = {"type": "function", "function": { + "name": "predict_fault", + "description": "Predict the most likely fault type of a machine from its current sensor readings, " + "using a trained classifier. Use this when a machine has a warning or fault.", + "parameters": {"type": "object", + "properties": {"machine_id": {"type": "string", "description": "Machine ID such as CNC-07"}}, + "required": ["machine_id"]}}} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/fl.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/fl.py new file mode 100644 index 000000000..4f93ce989 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/fl.py @@ -0,0 +1,68 @@ +"""Small helpers shared by every challenge: find the local Foundry Local endpoint and load models.""" +import json +import os +import shutil +import subprocess +from pathlib import Path + +from openai import OpenAI + +def _find_data(): + """Find resources/data by walking up from this file, so the same helper works in any lab sub-folder.""" + if os.environ.get("LAB_DATA"): + return Path(os.environ["LAB_DATA"]) + for parent in Path(__file__).resolve().parents: + if (parent / "resources" / "data").is_dir(): + return parent / "resources" / "data" + raise SystemExit("Could not find resources/data. Set the LAB_DATA environment variable to its path.") + + +DATA = _find_data() + + +def _foundry(*args, as_json=True): + exe = shutil.which("foundry") + if exe is None: + raise SystemExit("The 'foundry' command was not found. Install Foundry Local first (challenge 1).") + cmd = [exe, *args] + (["-o", "json"] if as_json else []) + out = subprocess.run(cmd, capture_output=True, text=True, encoding="utf-8", errors="replace") + if out.returncode != 0: + raise SystemExit(f"'{' '.join(cmd)}' failed:\n{out.stdout}{out.stderr}") + return json.loads(out.stdout) if as_json else out.stdout + + +def base_url(): + """Return the OpenAI-compatible URL of the local service, starting the service if needed.""" + if os.environ.get("FOUNDRY_LOCAL_URL"): + return os.environ["FOUNDRY_LOCAL_URL"].rstrip("/") + status = _foundry("server", "status") + if not status.get("running"): + _foundry("server", "start", as_json=False) + status = _foundry("server", "status") + return status["webUrls"][0].rstrip("/") + "/v1" + + +def client(): + # The local service ignores the key, but the OpenAI library requires a value. + return OpenAI(base_url=base_url(), api_key="local") + + +SERVED = Path(os.environ.get("FOUNDRY_SERVED_MODELS", Path.home() / ".foundry" / "served_models.json")) + + +def loaded_models(): + # A service started by serve_offline.py lists what it serves in a small file (the CLI cannot see that service). + if os.environ.get("FOUNDRY_LOCAL_URL") and SERVED.exists(): + return {a: {"alias": a, "id": i, "device": "see serve_offline.py", "fileSizeMb": 0} + for a, i in json.loads(SERVED.read_text(encoding="utf-8")).items()} + return {m["alias"]: m for m in _foundry("model", "list", "--loaded")["models"]} + + +def load(alias): + """Load a model (downloading it first if needed) and return the model id the service expects.""" + if alias not in loaded_models(): + if os.environ.get("FOUNDRY_LOCAL_URL"): + raise SystemExit(f"'{alias}' is not served. Start serve_offline.py with this alias.") + _foundry("model", "download", alias, as_json=False) + _foundry("model", "load", alias, as_json=False) + return loaded_models()[alias]["id"] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/hello.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/hello.py new file mode 100644 index 000000000..42bb7e339 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/hello.py @@ -0,0 +1,16 @@ +"""Challenge 1: your first call to a model that runs on your own machine.""" +import time + +import fl + +model = fl.load("phi-4-mini") +client = fl.client() + +start = time.perf_counter() +reply = client.chat.completions.create( + model=model, + messages=[{"role": "user", "content": "In one sentence: what does a technician check first when a CNC spindle vibrates?"}], + max_tokens=80, +) +print(reply.choices[0].message.content) +print(f"\nAnswered in {time.perf_counter() - start:.1f} s by {model} on this machine.") diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/offline_check.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/offline_check.py new file mode 100644 index 000000000..7843beebb --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/offline_check.py @@ -0,0 +1,38 @@ +"""Challenge 6: check that the copilot really works without the internet.""" +import json +import socket +import time +import urllib.parse + +import fl + + +def internet_reachable(host="1.1.1.1", port=443, timeout=3): + try: + socket.create_connection((host, port), timeout=timeout).close() + return True + except OSError: + return False + + +def main(): + url = fl.base_url() + host = urllib.parse.urlparse(url).hostname + print(f"Model service : {url}") + print(f"Runs locally : {'yes' if host in ('127.0.0.1', 'localhost', '::1') else 'NO - this is not a loopback address'}") + online = internet_reachable() + print(f"Internet : {'REACHABLE - disconnect (or block) the network and run this again' if online else 'not reachable (offline)'}") + + client = fl.client() + model = fl.load("phi-4-mini") + timings = [] + for question in ("In one sentence, what causes spindle bearing wear?", "Say OK."): + start = time.perf_counter() + client.chat.completions.create(model=model, max_tokens=40, messages=[{"role": "user", "content": question}]) + timings.append(round(time.perf_counter() - start, 2)) + print(f"Local answers : {timings} s") + print(json.dumps({"offline": not online, "local_latency_s": timings})) + + +if __name__ == "__main__": + main() diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/retrieval.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/retrieval.py new file mode 100644 index 000000000..19bde7fec --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/retrieval.py @@ -0,0 +1,135 @@ +"""Challenge 4: agentic retrieval over the local SOPs, with citations.""" +import json +import re +import sys +from pathlib import Path + +import numpy as np + +import fl + +EMBED_MODEL = "qwen3-embedding-0.6b" +CHAT_MODEL = "phi-4-mini" +INDEX = Path(__file__).resolve().parent / "sop_index.json" + +ANSWER_SYSTEM = ( + "You answer maintenance questions using ONLY the numbered SOP excerpts provided. " + "Cite the SOP ID in square brackets after each fact, for example [SOP-CNC-001]. " + 'If the excerpts do not contain the answer, reply exactly: "I could not find this in the SOPs."' +) + + +def chunk_sops(): + """One chunk per section (## heading). The SOP ID and revision travel with every chunk.""" + chunks = [] + for path in sorted((fl.DATA / "sops").glob("*.md")): + text = path.read_text(encoding="utf-8") + sop_id = path.name.split("_")[0] + revision = re.search(r"\*\*Revision:\*\*\s*(.+)", text) + header = text.split("\n## ")[0] + for section in text.split("\n## ")[1:]: + title, _, body = section.partition("\n") + chunks.append({ + "sop": sop_id, "revision": revision.group(1).split()[0] if revision else "?", + "section": title.strip(), + # Prefixing the header makes the chunk self-contained, so retrieval can match the machine and error code. + "text": f"{header.splitlines()[0]} ({sop_id})\n{header.split(chr(10), 1)[1].strip()}\n\n## {title}\n{body}".strip(), + }) + return chunks + + +def embed(client, model, texts, batch=16): + out = [] + for i in range(0, len(texts), batch): + out += [d.embedding for d in client.embeddings.create(model=model, input=texts[i:i + batch]).data] + v = np.array(out, dtype=np.float32) + return v / np.linalg.norm(v, axis=1, keepdims=True) + + +def build_index(client, embed_model): + chunks = chunk_sops() + vectors = embed(client, embed_model, [c["text"] for c in chunks]) + INDEX.write_text(json.dumps({"chunks": chunks, "vectors": vectors.tolist()}), encoding="utf-8") + return chunks, vectors + + +def load_index(): + data = json.loads(INDEX.read_text(encoding="utf-8")) + return data["chunks"], np.array(data["vectors"], dtype=np.float32) + + +def retrieve(client, embed_model, chunks, vectors, question, k=3): + q = embed(client, embed_model, [question])[0] + scores = vectors @ q + top = np.argsort(-scores)[:k] + return [(chunks[i], float(scores[i])) for i in top] + + +def decompose(client, chat_model, question): + """Split a compound question into one search query per topic. Small models do this plain-text task well.""" + reply = client.chat.completions.create( + model=chat_model, temperature=0, max_tokens=120, + messages=[{"role": "system", "content": + "Split the question into separate search queries, one per topic. " + 'Reply with only a JSON list of short strings, for example ["torque for bolts", "lockout steps"]. ' + "If the question has one topic, reply with a list of one string."}, + {"role": "user", "content": question}], + ).choices[0].message.content or "" + match = re.search(r"\[.*\]", reply, re.S) + try: + queries = [q for q in json.loads(match.group(0)) if isinstance(q, str) and q.strip()] if match else [] + except ValueError: + queries = [] + return queries[:3] or [question] + + +def search_tool(client, chat_model, embed_model, chunks, vectors, k_per_topic=3): + """Wrap retrieval as an agent tool. A compound query is split into topics, and every topic is searched.""" + def search_sops(query: str) -> dict: + queries = decompose(client, chat_model, query) + excerpts, seen = [], set() + for q in queries: + for c, s in retrieve(client, embed_model, chunks, vectors, q, k_per_topic): + key = (c["sop"], c["section"]) + if s >= 0.35 and key not in seen: + seen.add(key) + excerpts.append({"sop": c["sop"], "revision": c["revision"], "section": c["section"], "text": c["text"]}) + return {"queries": queries, "excerpts": excerpts or "No relevant SOP found."} + + schema = {"type": "function", "function": { + "name": "search_sops", + "description": "Search the maintenance SOPs (procedures, limits, torque values, intervals, error codes, " + "safety rules). It splits a question with several topics for you.", + "parameters": {"type": "object", + "properties": {"query": {"type": "string", "description": + "The technician's full question, copied word for word with every topic in it"}}, + "required": ["query"]}}} + return schema, search_sops + + +def answer(client, chat_model, embed_model, chunks, vectors, question, k=3, min_score=0.35): + hits = [(c, s) for c, s in retrieve(client, embed_model, chunks, vectors, question, k) if s >= min_score] + if not hits: + return "I could not find this in the SOPs.", [] + context = "\n\n".join(f"[{c['sop']} rev {c['revision']} / {c['section']}]\n{c['text']}" for c, _ in hits) + reply = client.chat.completions.create( + model=chat_model, temperature=0, max_tokens=250, + messages=[{"role": "system", "content": ANSWER_SYSTEM}, + {"role": "user", "content": f"SOP excerpts:\n{context}\n\nQuestion: {question}"}], + ).choices[0].message.content + return reply.strip(), [(c["sop"], c["section"], round(s, 2)) for c, s in hits] + + +if __name__ == "__main__": + client = fl.client() + embed_model = fl.load(EMBED_MODEL) + chat_model = fl.load(CHAT_MODEL) + if not INDEX.exists(): + chunks, vectors = build_index(client, embed_model) + print(f"Indexed {len(chunks)} chunks.") + else: + chunks, vectors = load_index() + question = " ".join(sys.argv[1:]) or "At what vibration level must a CNC spindle be stopped?" + text, sources = answer(client, chat_model, embed_model, chunks, vectors, question) + print(text) + print("Sources:", sources) diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/serve_offline.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/serve_offline.py new file mode 100644 index 000000000..6adb9c9a7 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/serve_offline.py @@ -0,0 +1,108 @@ +"""Start Foundry Local so that it does NOT need the internet to start (challenge 6). + +The `foundry` service loads its model catalog from the internet every time it starts. If it is restarted while +the device is offline, it cannot find any model, even models that are already on the disk. This script starts the +service from the Foundry Local Python SDK instead, with a catalog that is never contacted. It finds the models +in your local cache and serves them on the same kind of OpenAI-compatible address. + + python serve_offline.py serves phi-4-mini and qwen3-embedding-0.6b (CPU variants) on port 5272 + python serve_offline.py phi-4-mini --port 5300 + python serve_offline.py --accelerate use the GPU/NPU variants instead (see the note below) + +In another terminal, point the lab code at it: + + set FOUNDRY_LOCAL_URL=http://127.0.0.1:5272/v1 (PowerShell: $env:FOUNDRY_LOCAL_URL = "http://127.0.0.1:5272/v1") + +Download the models first, while you are online (challenge 1). Use `pip install foundry-local-sdk` before you disconnect. + +Note on GPU and NPU (--accelerate): these model variants need an "execution provider". The NPU one is installed with +Foundry Local. The GPU one (WebGPU) is downloaded the first time you run a GPU model online and then kept in +~/.foundry/ep. The SDK tries to download it again at every start, which fails offline and leaves every GPU model +unloadable. This script therefore points the SDK at the copy on the disk (FOUNDRY_LOCAL_WEBGPU_EP_LIBRARY). +Run a GPU model once while online before you rely on this. The CPU variants need no provider. +""" +import argparse +import json +import os +import time +from pathlib import Path + +from foundry_local_sdk import Configuration, FoundryLocalManager, LogLevel + +SERVED = Path(os.environ.get("FOUNDRY_SERVED_MODELS", Path.home() / ".foundry" / "served_models.json")) +LOOPBACK_CATALOG = ("http://127.0.0.1:9", None) # nothing listens here, so the catalog fails at once instead of timing out +WEBGPU_BUNDLES = Path.home() / ".foundry" / "ep" / "webgpu-ep" / "bundles" + + +def use_local_webgpu_provider(): + """Tell the SDK to use the WebGPU provider that is already on the disk. + + Without this the SDK tries to download the provider again every time it starts. Offline that fails, the provider is + not registered, and every GPU model refuses to load (the SDK still says 'success'). It was downloaded the first time + you ran a GPU model online. Returns the path that was used, or None. + """ + found = sorted(WEBGPU_BUNDLES.glob("*/bin/onnxruntime_providers_webgpu.dll")) if WEBGPU_BUNDLES.exists() else [] + if found and not os.environ.get("FOUNDRY_LOCAL_WEBGPU_EP_LIBRARY"): + os.environ["FOUNDRY_LOCAL_WEBGPU_EP_LIBRARY"] = str(found[-1]) + return os.environ.get("FOUNDRY_LOCAL_WEBGPU_EP_LIBRARY") + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("aliases", nargs="*", default=["phi-4-mini", "qwen3-embedding-0.6b"]) + ap.add_argument("--port", type=int, default=5272) + ap.add_argument("--cache", default=os.environ.get("FOUNDRY_CACHE", str(Path.home() / ".foundry" / "cache" / "models"))) + ap.add_argument("--accelerate", action="store_true", help="use the GPU/NPU variants (they must have been used once while online)") + args = ap.parse_args() + + if args.accelerate: + provider = use_local_webgpu_provider() + print(f"WebGPU provider: {provider or 'none on this machine (GPU variants will not load; NPU and CPU still work)'}") + + config = Configuration( + app_name="offline-copilot", model_cache_dir=args.cache, log_level=LogLevel.WARNING, + catalog_urls=[LOOPBACK_CATALOG], web=Configuration.WebService(urls=f"http://127.0.0.1:{args.port}")) + FoundryLocalManager.initialize(config) + manager = FoundryLocalManager.instance + + if args.accelerate: + try: + result = manager.download_and_register_eps(progress_callback=lambda *a: None) + print(f"execution providers registered: {result.registered_eps}") + except Exception as e: # carry on: the CPU variants need no provider + print(f"Could not register GPU/NPU providers ({str(e)[:80]}). Using CPU variants.") + + cached = manager.catalog.get_cached_models() + served = {} + for alias in args.aliases: + variants = [v for v in cached if v.alias == alias] + # try the preferred hardware first, then fall back to the other + variants.sort(key=lambda v: 0 if (("cpu" in v.id.lower()) != args.accelerate) else 1) + for variant in variants: + try: + variant.load() + served[alias] = variant.id + print(f"loaded {alias:<24} -> {variant.id}") + break + except Exception as e: + print(f" could not load {variant.id}: {str(e)[:100]}") + else: + raise SystemExit(f"'{alias}' is not in the local cache ({args.cache}). Download it first: foundry model download {alias}") + + manager.start_web_service() + SERVED.parent.mkdir(parents=True, exist_ok=True) + SERVED.write_text(json.dumps(served, indent=2), encoding="utf-8") + url = manager.urls[0].rstrip("/") + "/v1" + print(f"\nServing on {url}\nIn another terminal: $env:FOUNDRY_LOCAL_URL = \"{url}\"\nPress Ctrl+C to stop.") + try: + while True: + time.sleep(3600) + except KeyboardInterrupt: + pass + finally: + manager.stop_web_service() + SERVED.unlink(missing_ok=True) + + +if __name__ == "__main__": + main() diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/tool_router.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/tool_router.py new file mode 100644 index 000000000..b7ff16bd1 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/tool_router.py @@ -0,0 +1,56 @@ +"""Tool shortlisting for the copilot. + +A small model picks the wrong tool more often the more tools it is shown. So before the model sees a request, +we match the request against a short "card" of every tool with the embedding model that is already loaded for the +SOP search, and only show the model the best matches. (This is the same idea as retrieval over documents, applied +to tools. Research on tool retrieval reports large gains in tool-selection accuracy for this reason.) +""" +import re + +import retrieval + +# What each tool is for, in plain words, plus a few example requests. Examples matter most for a small embedding model. +CARDS = { + "get_machine_status": ( + "Look up the live status and current sensor readings of one machine: running or stopped, temperature, " + "vibration, load, current, error code.\n" + "How is CNC-03 doing? Read me the temperature of PRESS-02. Is the lathe LATHE-05 online? " + "What is the current load on ROB-04?"), + "predict_fault": ( + "Predict or diagnose the most likely fault of one machine with the trained fault classification model: " + "bearing wear, overheating, low lubrication.\n" + "Predict what is going to fail on CNC-07. Classify the fault state of CMP-01. " + "What is wrong with LATHE-05 according to the trained model? Run the diagnosis for PRESS-02."), + "search_sops": ( + "Search the maintenance procedures (SOPs): limits, torque values, intervals, error codes, safety rules, " + "how to do a task. General questions that name no machine.\n" + "What is the torque for the gearbox cover? How do I replace the filter? Explain error code E-XYZ-001. " + "What is the safety rule for working on a press? What does the procedure say? " + "ROB-04 reports error E-WLD-120, what is the procedure to fix it?"), + "create_maintenance_ticket": ( + "Open, create or log a maintenance ticket, work order or service request for a machine.\n" + "Open a ticket for CNC-03. Create a maintenance request for ROB-04. Log this problem as a ticket. " + "Write up a work order for the lathe."), +} +QUERY_PREFIX = "Instruct: Given a technician's request, find the tool that should handle it\nQuery: " +MACHINE_ID = re.compile(r"\b[A-Z]{2,6}-\d{2}\b", re.I) # CNC-07, PRESS-02 ... (error codes like E-SPN-214 do not match) +NEEDS_MACHINE_ID = {"get_machine_status", "predict_fault", "create_maintenance_ticket"} + + +class ToolRouter: + def __init__(self, client, embed_model, names=None, prefix=QUERY_PREFIX): + self.client, self.embed_model, self.prefix = client, embed_model, prefix + self.names = list(names or CARDS) + self.vectors = retrieval.embed(client, embed_model, [CARDS[n] for n in self.names]) + + def rank(self, request): + q = retrieval.embed(self.client, self.embed_model, [self.prefix + request])[0] + scores = self.vectors @ q + return sorted(zip(self.names, scores.tolist()), key=lambda x: -x[1]) + + def candidates(self, request): + """Tool names for this request, best first. A tool that needs a machine ID is dropped when the request names none.""" + ranked = self.rank(request) + if not MACHINE_ID.search(request): + ranked = [(n, s) for n, s in ranked if n not in NEEDS_MACHINE_ID] + return [n for n, _ in ranked] diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/tools.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/tools.py new file mode 100644 index 000000000..5f05854db --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/tools.py @@ -0,0 +1,57 @@ +"""The two tools the maintenance copilot can call.""" +import json +from datetime import datetime, timezone +from pathlib import Path + +from fl import DATA + +TICKETS = Path(__file__).resolve().parent / "tickets.jsonl" + + +def get_machine_status(machine_id: str) -> dict: + machines = json.loads((DATA / "machines.json").read_text(encoding="utf-8"))["machines"] + for m in machines: + if m["id"].lower() == machine_id.strip().lower(): + return m + return {"error": f"Unknown machine '{machine_id}'", "known_machines": [m["id"] for m in machines]} + + +def create_maintenance_ticket(machine_id: str, summary: str, priority: str) -> dict: + if priority not in ("low", "medium", "high"): + return {"error": "priority must be low, medium or high"} + if "error" in get_machine_status(machine_id): + return {"error": f"Unknown machine '{machine_id}'"} + existing = TICKETS.read_text(encoding="utf-8").splitlines() if TICKETS.exists() else [] + ticket = { + "ticket_id": f"MT-{1001 + len(existing)}", + "machine_id": machine_id.upper(), + "summary": summary, + "priority": priority, + "created_utc": datetime.now(timezone.utc).isoformat(timespec="seconds"), + } + with TICKETS.open("a", encoding="utf-8") as f: + f.write(json.dumps(ticket) + "\n") + return ticket + + +TOOL_SCHEMAS = [ + {"type": "function", "function": { + "name": "get_machine_status", + "description": "Get the live status and sensor readings of one machine, for example CNC-07 or PRESS-02.", + "parameters": {"type": "object", + "properties": {"machine_id": {"type": "string", "description": "Machine ID such as CNC-07"}}, + "required": ["machine_id"]}}}, + {"type": "function", "function": { + "name": "create_maintenance_ticket", + "description": "Create a maintenance ticket for a machine.", + "parameters": {"type": "object", + "properties": {"machine_id": {"type": "string", "description": "Machine ID such as CNC-07"}, + "summary": {"type": "string", "description": "Short description of the problem"}, + "priority": {"type": "string", "enum": ["low", "medium", "high"]}}, + "required": ["machine_id", "summary", "priority"]}}}, +] + +IMPLEMENTATIONS = { + "get_machine_status": get_machine_status, + "create_maintenance_ticket": create_maintenance_ticket, +} diff --git a/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/train_fault_classifier.py b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/train_fault_classifier.py new file mode 100644 index 000000000..e6b9ff287 --- /dev/null +++ b/03-Azure/01-04-AI/10_Offline_Industrial_Copilot/walkthrough/code/train_fault_classifier.py @@ -0,0 +1,86 @@ +"""Challenge 5: train a fault classifier and export it to ONNX. + +Run it locally (it takes a few seconds) or paste it into an Azure Machine Learning notebook. + + python train_fault_classifier.py [output_folder] [sensor_history.csv] + +By default it writes an 'artifacts' folder next to this file and reads resources/data/sensor_history.csv. +""" +import csv +import json +import sys +from pathlib import Path + +import numpy as np +from skl2onnx import to_onnx +from sklearn.ensemble import RandomForestClassifier +from sklearn.metrics import accuracy_score, confusion_matrix +from sklearn.model_selection import train_test_split + +HERE = Path(__file__).resolve().parent +DEFAULT_CSV = next((p / "resources" / "data" / "sensor_history.csv" for p in HERE.parents + if (p / "resources" / "data").is_dir()), HERE / "sensor_history.csv") +OUT = Path(sys.argv[1]) if len(sys.argv) > 1 else HERE / "artifacts" +CSV = Path(sys.argv[2]) if len(sys.argv) > 2 else DEFAULT_CSV +FEATURES = ["vibration_mm_s", "temperature_c", "load_pct", "current_a", "hours_since_service"] +LABELS = ["normal", "bearing_wear", "overheating", "low_lubrication"] + + +def generate(rows_per_class=600, seed=42): + rng = np.random.default_rng(seed) + n = rows_per_class + parts = [] + for label, (vib, temp, hours) in { + "normal": ((2.5, 0.9), (60, 6), (0, 2500)), + "bearing_wear": ((7.8, 1.6), (73, 6), (1200, 3500)), + "overheating": ((3.4, 1.0), (92, 7), (500, 3500)), + "low_lubrication": ((5.4, 0.8), (66, 5), (2000, 4200)), + }.items(): + load = rng.uniform(30, 90, n) + current = np.clip(load * 0.36 + rng.normal(0, 2, n) + (6 if label == "overheating" else 0), 1, 80) + features = np.column_stack([ + np.clip(rng.normal(*vib, n), 0.2, 20), + np.clip(rng.normal(*temp, n), 20, 130), + load, + current, + rng.uniform(*hours, n), + ]).round(1) + parts.append(np.column_stack([features, np.full(n, LABELS.index(label))])) + data = np.vstack(parts) + rng.shuffle(data) + return data + + +def write_csv(data): + CSV.parent.mkdir(parents=True, exist_ok=True) + with CSV.open("w", newline="", encoding="utf-8") as f: + w = csv.writer(f) + w.writerow(FEATURES + ["label"]) + for row in data: + w.writerow([*row[:-1], LABELS[int(row[-1])]]) + + +def load_csv(): + with CSV.open(encoding="utf-8") as f: + rows = list(csv.DictReader(f)) + x = np.array([[float(r[c]) for c in FEATURES] for r in rows], dtype=np.float32) + y = np.array([LABELS.index(r["label"]) for r in rows]) + return x, y + + +if __name__ == "__main__": + if not CSV.exists(): + write_csv(generate()) + x, y = load_csv() + x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=1, stratify=y) + model = RandomForestClassifier(n_estimators=40, max_depth=6, random_state=1).fit(x_train, y_train) + pred = model.predict(x_test) + print(f"rows={len(x)} test accuracy={accuracy_score(y_test, pred):.3f}") + print(LABELS) + print(confusion_matrix(y_test, pred)) + + OUT.mkdir(parents=True, exist_ok=True) + onnx_model = to_onnx(model, x[:1], options={id(model): {"zipmap": False}}, target_opset=17) + (OUT / "fault_classifier.onnx").write_bytes(onnx_model.SerializeToString()) + (OUT / "fault_labels.json").write_text(json.dumps({"features": FEATURES, "labels": LABELS}, indent=2)) + print("wrote", OUT / "fault_classifier.onnx", f"({(OUT / 'fault_classifier.onnx').stat().st_size // 1024} KB)") diff --git a/03-Azure/01-04-AI/Readme.md b/03-Azure/01-04-AI/Readme.md index ecd5c94d2..ba31c229b 100644 --- a/03-Azure/01-04-AI/Readme.md +++ b/03-Azure/01-04-AI/Readme.md @@ -14,6 +14,8 @@ A collection of MicroHacks focused on AI on Azure and the GitHub + Azure agentic * [**Predictive Maintenance Multi-Agent System**](08_Predictive_Maintenance/README.md) — Build an end-to-end multi-agent predictive maintenance workflow with **Microsoft Foundry**, **Microsoft Agent Framework**, **.NET Aspire**, and **Azure Cosmos DB**: classify telemetry anomalies, diagnose root causes via **Foundry IQ** knowledge bases and **API Management** MCP tools, plan repairs in C#, and orchestrate cross-language A2A agents into a sequential autonomous maintenance pipeline. * [**Air-Gapped Agentic AI: Orchestrating Local SLMs and IoT on Edge Kubernetes**](09_Air_Gapped_Agentic_Edge_AI/Readme.md) — Build an end-to-end, disconnected sovereign edge AI and IoT workflow on **Azure Arc-enabled Kubernetes** (K3s on Ubuntu edge host): ingest and filter manufacturing sensor telemetry with **Azure IoT Operations** (MQTT Broker & Dataflows), serve a quantized CPU Small Language Model via **Foundry Local** (`Microsoft.Foundry` Arc extension + Gateway API/Istio + `onnx-genai`), authenticate in-cluster edge agents using **Kubernetes Service Account Tokens (SAT)**, and prove autonomous closed-loop actuation under strict air-gapped network policies. +* [**Offline Industrial Copilot**](10_Offline_Industrial_Copilot/Readme.md): Build a maintenance technician copilot for a fictional manufacturer that runs **entirely offline** with **Foundry Local**. Choose a small language model for the device, give the agent local tools through function calling (machine status, maintenance tickets, SOP search with cited sources), add agentic retrieval over local knowledge, train a fault classifier in **Azure Machine Learning** and run it at the edge as ONNX, then pull the network cable and prove every step still works. Basic Python; no Kubernetes or Azure Arc needed. + * [**Regulatory Intelligence: From Notice to Action**](11_Healthcare_Regulatory_Intelligence/README.md): Keep a fictional life-sciences company on top of regulatory change. Crawl health-authority and payer portals into a regulatory lake, ground **Microsoft Foundry** prompt agents in it with **Foundry IQ**, cite exact paragraphs, link obligations to products, labels, studies, SOPs and quality records, answer staff questions, and brief leadership by region and portfolio, with **human sign-off** and an audit trail. Stretch: an event-driven agent chain with the **Microsoft Agent Framework** in GitHub Codespaces. Collaborators wanted :) — see [CONTRIBUTING.md](../../CONTRIBUTING.md) to add your own. \ No newline at end of file