Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{
"name": "Azure / AI / Contact Center",
"image": "mcr.microsoft.com/devcontainers/python:3.12",
"features": {
"ghcr.io/devcontainers/features/azure-cli:1": {}
},
"workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}/03-Azure/01-04-AI/12_Contact_Center",
"remoteUser": "vscode",
"waitFor": "postCreateCommand",
"postCreateCommand": "pip install --user pytest && python3 -m pytest -q",
"forwardPorts": [
8000
],
"portsAttributes": {
"8000": {
"label": "Contact center chat page",
"visibility": "private",
"onAutoForward": "openBrowser"
}
},
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance"
],
"settings": {
"python.testing.pytestEnabled": true,
"python.testing.pytestArgs": [
"tests"
],
"terminal.integrated.defaultProfile.linux": "bash",
"git.openRepositoryInParentFolders": "always"
}
}
}
}
126 changes: 126 additions & 0 deletions 03-Azure/01-04-AI/12_Contact_Center/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Contact Center Agents: Voice-First Customer Service with Microsoft Foundry

- [**MicroHack introduction**](#microhack-introduction)
- [**What you'll build**](#what-youll-build)
- [**MicroHack context**](#microhack-context)
- [**Objectives**](#objectives)
- [**MicroHack challenges**](#microhack-challenges)
- [**Contributors**](#contributors)

## MicroHack introduction

Customers call when something matters to them: a bill they don't understand, a service that has stopped, a card they've lost. They want a fast answer, the right action and, when it gets difficult, a person who already knows the story. **Contact Center Agents** builds that experience with agents in Microsoft Foundry.

In one day you build a **voice agent** that answers from your company's knowledge articles, checks who it is talking to and takes real actions in a CRM. You add guardrails, hand difficult calls to a human with a summary, give that human an **agent-assist** copilot, analyse recorded calls and spot trends as a supervisor. Then you measure how good your agent really is.

Everything happens in the browser: the agents in the **Microsoft Foundry portal**, one setup command in **Azure Cloud Shell**. You don't write code. The optional last challenge publishes your agent to a web page, with a little code in GitHub Codespaces.

> [!TIP]
> You talk to your agent with your own voice, so bring a headset with a microphone, or use a quiet room.

## Pick your industry

The contact center works the same way in every industry, so **you choose the company you work for** in Challenge 1. Each industry pack has its own knowledge articles, customers, appointments, recorded calls and test questions:

| Pack | Company | What customers call about |
|---|---|---|
| `telco` | **Kestrel Connect** | Broadband outages, engineer visits, bills, moving home |
| `energy` | **Lumora Energy** | Estimated bills, smart meters, payment difficulties, power cuts |
| `bank` | **Harbourline Bank** | Lost cards, disputed payments, scams, mortgage appointments |
| `insurance` | **Northmere Insurance** | Storm damage, motor claims, assessor visits |
| `retail` | **Parcelpine** | Late deliveries, returns, damaged items |

Every company, customer, address and phone call is fictional. Postcodes use the unused `ZZ` area.

## What you'll build

| Challenge | Capability | Built with | What it produces |
|---|---|---|---|
| 1 | **Your company and its knowledge** | Supplied tool in Cloud Shell, Azure Blob Storage, **Foundry IQ** | Your industry pack loaded, and a knowledge base of articles your agents can cite |
| 2 | **Voice self-service agent** | Foundry **voice agent**, Foundry IQ | An agent you talk to that answers from the knowledge base, and a chat version of it |
| 3 | **Take action** | **OpenAPI tool** connected to the mock CRM | The agent checks who is calling, looks up the account, books appointments and opens cases |
| 4 | **Safety and compliance** | Foundry **guardrails**, instructions | Content filters, Prompt Shields and a blocklist; personal data handled safely; vulnerable customers recognised |
| 5 | **Handoff and agent assist** | OpenAPI tool, a second Foundry agent | Difficult calls go to a human with a summary, and the human gets suggested replies and the next best action |
| 6 | **Post-contact analytics and supervisor insights** | **Content Understanding**, Foundry IQ | Summary, sentiment, intent and a quality scorecard for each recorded call, and trends across a week of contacts |
| 7 | **Measure quality** | Foundry **evaluations** | Scores for task adherence, intent resolution and tool use on your industry's test questions, compared across two settings |
| 8 ⭐ *(stretch)* | **Publish** | Foundry publishing, a small web page in Codespaces | Your voice agent as a shareable web app, and the chat agent on your own page in GitHub Codespaces |

![Contact center scenario: customers call or chat, the voice agent answers from your company's knowledge and acts in its systems, a human adviser takes over when needed, and every contact is analysed](images/readme-scenario.png)

### What each Azure service does here

| Service | Its job in this hack |
|---|---|
| **Microsoft Foundry Agent Service** | Hosts your agents: the voice agent, its chat twin, the agent-assist copilot and the supervisor agent. You can inspect every model and tool call in **Traces**. |
| **Voice agents** in Foundry | Turn a prompt agent into a real-time spoken conversation with speech recognition, a realtime model and a natural voice. |
| **Foundry IQ** (on **Azure AI Search**) | Indexes the knowledge articles and returns the best passages, with their source, so agents answer from your company's content instead of from memory. |
| **Azure Functions** (Flex Consumption) | Runs the mock CRM that holds customers, appointment slots, cases and handoffs. Your agents call it through an **OpenAPI tool**. |
| **Content Understanding** | Turns a recorded call into a transcript, summary, sentiment, intent and your own quality fields. |
| **Azure Blob Storage** | Holds the knowledge articles, the supervisor's contact records, the recorded calls and the CRM's data. |

![Contact center architecture: channels, the setup tool in Cloud Shell, four Foundry agents with guardrails and Content Understanding, Foundry IQ on Azure AI Search, Blob Storage, the mock CRM on Azure Functions and the optional chat page in Codespaces](images/readme-architecture.png)

The editable sources are [readme-scenario.excalidraw](images/readme-scenario.excalidraw) and [readme-architecture.excalidraw](images/readme-architecture.excalidraw). Open them at <https://excalidraw.com> to change them.

Not everything is an AI job. **Checking identity, booking a slot and recording a case are ordinary code** in the CRM: they must give the same answer every time. **Listening, explaining and deciding what to do next are agent jobs.** And when a customer is vulnerable or asks for a person, **a human takes over**.

## MicroHack context

### The problem today

- Customers wait in queues for simple answers that are already in a knowledge article.
- Phone menus and scripts can't handle the way people actually talk.
- When a call is handed over, the customer has to explain everything again.
- Quality checks cover a tiny sample of calls, and trends are spotted weeks late.

### Why it matters

A good contact center agent resolves routine requests at any hour, takes real action instead of just answering, recognises when a person is needed and hands over without making the customer repeat themselves. Supervisors see every contact, not a sample. In the finish challenge you estimate what that could mean for your company.

## Objectives

By the end of the day you can:

- Build a Foundry **voice agent** and ground it in a **Foundry IQ** knowledge base with citations.
- Connect an agent to a business system with an **OpenAPI tool**, and make it check identity before acting.
- Protect an agent with **guardrails**, and design for **vulnerable customers** and personal data.
- Hand a conversation to a human with full context, and build an **agent-assist** copilot.
- Analyse recorded calls with **Content Understanding** and find trends as a supervisor.
- **Evaluate** an agent on a test set and compare settings with evidence.
- *(Stretch)* Publish an agent to the web.

## MicroHack challenges

### Prerequisites

- A modern browser (Edge or Chrome) with **microphone access**, signed in with the lab account from your facilitator. A headset is recommended.
- Access to the [Microsoft Foundry portal](https://ai.azure.com), the [Azure portal](https://portal.azure.com) and Azure Cloud Shell.
- Your **lab dashboard** values: `ResourceGroupName`, `FoundryProjectEndpoint`, `ModelDeploymentName`, `EmbeddingDeploymentName`, `AnalysisModelDeploymentName`, `StorageAccountName`, `SearchServiceName` and `CrmUrl`.
- No local software installation. For the stretch challenge, a GitHub account for Codespaces.

> [!NOTE]
> Your Foundry project, model deployments, storage account, Azure AI Search service and the mock CRM are deployed for you by the [lab automation](labautomation/README.md). You create everything else yourself.

### Challenges

| Challenge | Title | Duration |
|---|---|---|
| [Challenge 1](challenges/challenge-01.md) | Pick your company and build its knowledge base | 45 min |
| [Challenge 2](challenges/challenge-02.md) | Voice self-service agent | 45 min |
| [Challenge 3](challenges/challenge-03.md) | Take action with the CRM | 50 min |
| [Challenge 4](challenges/challenge-04.md) | Safety and compliance | 45 min |
| [Challenge 5](challenges/challenge-05.md) | Handoff and agent assist | 45 min |
| [Challenge 6](challenges/challenge-06.md) | Post-contact analytics and supervisor insights | 50 min |
| [Challenge 7](challenges/challenge-07.md) | Measure quality | 40 min |
| [Challenge 8 ⭐](challenges/challenge-08.md) | Publish your agent (stretch) | 30 to 45 min |
| [Finish](challenges/finish.md) | Estimate the impact and clean up | 15 min |

Solutions and expected results for coaches are in [walkthrough](walkthrough/).

## Contributors

| Name | Role |
|---|---|
| Digvijay Chauhan | Author & maintainer |

<!-- Add your name here when you contribute -->
118 changes: 118 additions & 0 deletions 03-Azure/01-04-AI/12_Contact_Center/challenges/challenge-01.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Challenge 1: Pick your company and build its knowledge base

**[Home](../README.md)** - [Next Challenge →](challenge-02.md)

## 🎯 Objective

Choose the company your contact center works for, load its knowledge articles, recorded calls and customer records into your lab, and build a **Foundry IQ** knowledge base so your agents can answer from it with citations.

## 🧭 Context

A contact center agent is only as good as what it knows. Each company has its own **knowledge articles**: how bills work, what to do in an outage, how to check who is calling, when to hand over to a person. Your agents must answer from these articles, not from general knowledge, and say where an answer came from.

Your lab already has a **mock CRM** with the customers, appointment slots and cases of all five companies. The setup command tells it which company you chose.

## ✅ Tasks

### Part A: Check your Foundry project (5 min)

1. Open [ai.azure.com](https://ai.azure.com) and make sure the **New Foundry** toggle at the top right is on.
2. Select **your** project from the project picker next to *Microsoft Foundry* at the top left. Its name is the last part of `FoundryProjectEndpoint` on your lab dashboard (`contact-center`).
3. Select **Build**, then **Models** in the left menu. On the **Deployments** tab, check that the chat model (`ModelDeploymentName`, default `gpt-5.4-mini`) and the embedding model (`EmbeddingDeploymentName`, default `text-embedding-3-large`) show **Succeeded**.

![Build, then Models, Deployments tab: gpt-5.4-mini and text-embedding-3-large with deployment status Succeeded](../images/challenge-01-model-deployments.png)

### Part B: Choose your company (10 min)

1. With your team, pick one industry. The rest of the day works the same for every choice.

| Industry | Company | Good if your team works in |
|---|---|---|
| `telco` | Kestrel Connect | Telecom, media, utilities with field engineers |
| `energy` | Lumora Energy | Energy, water, regulated utilities |
| `bank` | Harbourline Bank | Banking, payments, lending |
| `insurance` | Northmere Insurance | Insurance, claims, financial services |
| `retail` | Parcelpine | Retail, e-commerce, logistics |

2. In a private browser window, sign in to the [Azure portal](https://portal.azure.com) with your lab account. Select the **Cloud Shell** icon (`>_`) in the top bar and choose **Bash**. If you are asked about storage, choose the option without storage.

![The Cloud Shell icon in the Azure portal top bar](../images/challenge-01-open-cloud-shell.png)

3. Copy only this lab from GitHub and move into it:

```bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/microsoft/MicroHack contact-center-microhack
cd contact-center-microhack
git sparse-checkout set 03-Azure/01-04-AI/12_Contact_Center
cd 03-Azure/01-04-AI/12_Contact_Center
```

If your facilitator gives you a fork or a branch, use that repository URL and add `--branch <name>` after `--sparse`.

4. Check that Cloud Shell uses your lab subscription: `az account show --query name -o tsv`. If not, run `az account set --subscription "<lab subscription>"`.
5. Load your company. Replace the values with your choice and the values from your dashboard:

```bash
python3 -m contactcenter --account <StorageAccountName> setup --industry <telco|energy|bank|insurance|retail> --crm-url <CrmUrl>
```

The command replaces the contents of three storage containers with your company's files and switches the CRM to your company. It ends with a few **test customers**: keep them, you will use them to test your agents.

![Cloud Shell: the setup command loads 8 knowledge articles, 1 supervisor insights file and 3 recorded calls, the CRM check says OK, and test customers are listed](../images/challenge-01-setup.png)

> [!NOTE]
> Cloud Shell without storage starts empty every session. If it restarts, repeat step 3 to clone again. Your company's data is safe in your storage account.

### Part C: Look at your company (10 min)

1. Open your **CrmUrl** in a new browser tab. The CRM's home page shows your company and, later today, everything your agents do: appointments booked, cases opened and handoffs to people.
2. In the Azure portal, open your storage account (`StorageAccountName`), then **Storage browser**, then **Blob containers**:
- `knowledge`: your company's eight knowledge articles. Open one: every paragraph has a label such as `[KB-KC-005 p2]`, so an answer can point to the exact paragraph.
- `insights`: a week of analysed contacts, for the supervisor in Challenge 6.
- `calls`: three recorded calls, for Challenge 6.
3. With your team, read two articles that customers will ask about: `KB-…-001` (checking who you are speaking to) and `KB-…-002` (vulnerable customers). Your agents must follow both.

### Part D: Build the Foundry IQ knowledge base (20 min)

1. In the Foundry portal, select **Build**, then **Knowledge** in the left menu.
2. Connect Foundry IQ to your search service. Under **Foundry IQ resource**, select your existing service (`SearchServiceName`, for example `search-contact-…`). Do not create a new one.
3. Set **Auth Type** to **Project Managed Identity**, then select **Connect**.
4. On the **Knowledge bases** tab, select **Create a knowledge base**, and fill in:
- **Name:** `contact-knowledge-kb`; **Description:** for example *Customer service knowledge articles*.
- **Chat completions model:** your `ModelDeploymentName`.
- **Retrieval reasoning effort:** **Minimal**; **Output mode:** **Extractive data**.
5. Under **Knowledge sources**, select **Add sources**, then **Azure Blob Storage**, and fill in:

| Field | Value |
|---|---|
| Name | `contact-knowledge` |
| Description | *Customer service knowledge articles with paragraph labels: policies, processes, prices and service levels.* |
| Storage account | `StorageAccountName` |
| Container name | **`knowledge`** |
| Authentication type | **System assigned identity** |
| Content extraction mode | **Minimal** |
| Embedding model | `EmbeddingDeploymentName` |
| Chat completions model | leave empty |

![Create a knowledge source: contact-knowledge on the knowledge container with System assigned identity, Minimal extraction and text-embedding-3-large](../images/challenge-01-knowledge-source.png)

Select **Create**, then **Save knowledge base**. Within a few minutes the source changes from **Creating** to **Active**.

![The saved contact-knowledge-kb knowledge base with gpt-5.4-mini, Minimal reasoning effort, Extractive data and the contact-knowledge source Active](../images/challenge-01-knowledge-base.png)

> [!IMPORTANT]
> Use only the `knowledge` container here. The `insights` container holds internal contact records for the supervisor; customers must never hear them. You will give the supervisor its own knowledge base in Challenge 6.

## ✔️ Success criteria

- [ ] Your team chose an industry, and the setup command printed your company, the CRM check and test customers.
- [ ] Your CRM home page shows your company's name.
- [ ] `contact-knowledge-kb` exists with the `contact-knowledge` source **Active**, built from the `knowledge` container only.
- [ ] You can explain why the supervisor's contact records are kept out of the customer-facing knowledge base.

## 📚 Learning resources

- [What is Foundry IQ?](https://learn.microsoft.com/azure/foundry/agents/concepts/what-is-foundry-iq)
- [Azure Cloud Shell](https://learn.microsoft.com/azure/cloud-shell/overview)

[Coach solution and troubleshooting](../walkthrough/challenge-01/solution-01.md)
Loading
Loading