Skip to content
/ contoso-chat Public
forked from Azure-Samples/contoso-chat

This sample has the full End2End process of creating RAG application with Prompty and AI Studio. It includes GPT 3.5 Turbo LLM application code, evaluations, deployment automation with AZD CLI, GitHub actions for evaluation and deployment and intent mapping for multiple LLM task mapping.

License

MIT license
1 star 3.4k forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

HenriqueLee-git/contoso-chat

BranchesTags
 
 

Repository files navigation

page_type languages products urlFragment name description
sample
azdeveloper
python
bash
bicep
prompty
azure
azure-openai
azure-cognitive-search
azure-cosmos-db
contoso-chat
Contoso Chat - Retail RAG Copilot with Azure AI Studio and Prompty (Python Implementation)
Build, evaluate, and deploy, a RAG-based retail copilot that responds to customer questions with responses grounded in the retailer's product and customer data.

Warning

This sample is being actively updated at present and make have breaking changes. We are refactoring the code to use new Azure AI platform features and moving deployment from Azure AI Studio to Azure Container Apps. We will remove this notice once the migration is complete. Till then, please pause on submitting new issues as codebase is changing.

Some of the features used in this repository are in preview. Preview versions are provided without a service level agreement, and they are not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.**

Contoso Chat: Retail RAG Copilot with Azure AI Studio and Prompty

Open in GitHub Codespaces Open in Dev Containers

Table of Contents

Overview

Contoso Outdoor is an online retailer specializing in hiking and camping equipment for outdoor enthusiasts. The website offers an extensive catalog of products - resulting in customers needing product information and recommendations to assist them in making relevant purchases.

Contoso Outdoor

This sample implements Contoso Chat - a retail copilot solution for Contoso Outdoor that uses a retrieval augmented generation design pattern to ground chatbot responses in the retailer's product and customer data. Customers can now ask questions from the website in natural language, and get relevant responses along with potential recommendations based on their purchase history - with responsible AI practices to ensure response quality and safety.

Contoso Chat

The sample illustrates the end-to-end workflow (GenAIOps) for building a RAG-based copilot code-first with Azure AI and Prompty. By exploring and deploying this sample, you will learn to:

  1. Ideate and iterate rapidly on app prototypes using Prompty
  2. Deploy and use Azure OpenAI models for chat, embeddings and evaluation
  3. Use Azure AI Search (indexes) and Azure CosmosDB (databases) for your data
  4. Evaluate chat responses for quality using AI-assisted evaluation flows
  5. Host the application as a FastAPI endpoint deployed to Azure Container Apps
  6. Provision and deploy the solution using the Azure Developer CLI
  7. Support Responsible AI practices with content safety & assessments

Features

The project template provides the following features:

It also comes with:

  • Sample product and customer data for rapid prototyping
  • Sample application code for chat and evaluation workflows
  • Sample datasets and custom evaluators using prompty assets

Architecture Diagram

Architecture

Demo Video

(In Planning) - Get an intuitive sense for how simple it can be to go from template discovery, to codespaces launch, to application deployment with azd up. Watch this space for a demo video.

Versions

The Contoso Chat sample has undergone numerous architecture and tooling changes since its first version back in 2023. The table below links to legacy versions for awareness only. We recommend all users start with the latest version to leverage the latest tools and practices.

Version Description
v0 : #cc2e808 MSAITour 2023-24 (dag-flow, jnja template) - Skillable Lab
v1 : msbuild-lab322 MSBuild 2024 (dag-flow, jnja template) - Skillable Lab
v2 : raghack-24 RAG Hack 2024 (flex-flow, prompty asset) - AZD Template
v3 : main 🆕 MSAITour 2024-25 (prompty asset, ACA)- AZD Template

Pre-requisites

To deploy and explore the sample, you will need:

  1. An active Azure subscription - Signup for a free account here
  2. An active GitHub account - Signup for a free account here
  3. Access to Azure OpenAI Services - Learn about Limited Access here
  4. Access to Azure AI Search - With Semantic Ranker (premiun feature)
  5. Available Quota for: text-embedding-ada-002, gpt-35-turbo. and gpt-4

We recommend deployments to swedencentral or francecentral as regions that can support all these models. In addition to the above, you will also need the ability to:

  • provision Azure Monitor (free tier)
  • provision Azure Container Apps (free tier)
  • provision Azure CosmosDB for noSQL (free tier)

From a tooling perspective, familiarity with the following is useful:

  • Visual Studio Code (and extensions)
  • GitHub Codespaces and dev containers
  • Python and Jupyter Notebooks
  • Azure CLI, Azure Developer CLI and commandline usage

Getting Started

You have three options for setting up your development environment:

  1. Use GitHub Codespaces - for a prebuilt dev environment in the cloud
  2. Use Docker Desktop - for a prebuilt dev environment on local device
  3. Use Manual Setup - for control over all aspects of local env setup

We recommend going with GitHub Codespaces for the fastest start and lowest maintenance overheads. Pick one option below - click to expand the section and view the details.

1️⃣ | Quickstart with GitHub Codespaces
  1. Fork this repository to your personal GitHub account
  2. Click the green Code button in your fork of the repo
  3. Select the Codespaces tab and click Create new codespaces ...
  4. You should see: a new browser tab launch with a VS Code IDE
  5. Wait till Codespaces is ready - VS Code terminal has active cursor.
  6. ✅ | Congratulations! - Your Codespaces environment is ready!
2️⃣ | Get Started with Docker Desktop
  1. Install VS Code (with Dev Containers Extension) to your local device
  2. Install Docker Desktop to your local device - and start the daemon
  3. Fork this repository to your personal GitHub account
  4. Clone the fork to your local device - open with Visual Studio Code
  5. If Dev Containers Extension installed - you see: "Reopen in Container" prompt
  6. Else read Dev Containers Documentation - to launch it manually
  7. Wait till the Visual Studio Code environment is ready - cursor is active.
  8. ✅ | Congratulations! - Your Docker Desktop environment is ready!
3️⃣ | Get Started with Manual Setup
  1. Verify you have Python 3 installed on your machine
  2. Install dependencies with pip install -r src/api/requirements.txt
  3. Install the Azure Developer CLI for your OS
  4. Install the Azure CLI for your OS
  5. Verify that all required tools are installed and active: 
    az version
azd version
prompty --version
python --version

Once you have set up the development environment, it's time to get started with the development workflow by first provisioning the required Azure infrastructure, then deploying the application from the template.

Development

Regardless of the setup route you chose, you should at this point have a Visual Studio Code IDE running, with the required tools and package dependencies met in the associated dev environment.

1️⃣ | Authenticate With Azure

  1. Open a VS Code terminal and authenticate with Azure CLI. Use the --use-device-code option if authenticating from GitHub Codespaces. Complete the auth workflow as guided.

    az login --use-device-code

  2. Now authenticate with Azure Developer CLI in the same terminal. Complete the auth workflow as guided. You should see: Logged in on Azure.

    azd auth login
2️⃣ | Provision-Deploy with AZD

  1. Run azd up to provision infrastructure and deploy the application, with one command. (You can also use azd provision, azd deploy separately if needed)

    azd up

  2. The command will ask for an environment name, a location for deployment and the subscription you wish to use for this purpose.

    • The environment name maps to rg-ENVNAME as the resource group created
    • The location should be swedencentral or francecentral for model quota
    • The subscription should be an active subscription meeting pre-requistes

  3. The azd up command can take 15-20 minutes to complete. Successful completion sees a SUCCESS: ... messages posted to the console. We can now validate the outcomes.

3️⃣ | Validate the Infrastructure
  1. Visit the Azure Portal - look for the rg-ENVNAME resource group created above
  2. Click the Deployments link in the Essentials section - wait till all are completed.
  3. Return to Overview page - you should see: 35 deployments, 15 resources
  4. Click on the Azure CosmosDB resource in the list
    • Visit the resource detail page - click "Data Explorer"
    • Verify that it has created a customers database with data items in it
  5. Click on the Azure AI Search resource in the list
    • Visit the resource detail page - click "Search Explorer"
    • Verify that it has created a contoso-products index with data items in it
  6. Click on the Azure Container Apps resource in the list
    • Visit the resource detail page - click Application Url
    • Verify that you see a hosted endpoint with a Hello World message on page
  7. Next, visit the Azure AI Studio portal
    • Sign in - you should be auto-logged in with existing Azure credential
    • Click on All Resources - you should see an AIServices and Hub resources
    • Click the hub resource - you should see an AI Project resource listed
    • Click the project resource - look at Deployments page to verify models
  8. ✅ | Congratulations! - Your Azure project infrastructure is ready!
4️⃣ | Validate the Deployment
  1. The azd up process also deploys the application as an Azure Container App
  2. Visit the ACA resource page - click on Application Url to view endpoint
  3. Add a /docs suffix to default deployed path - to get a Swagger API test page
  4. Click Try it out to unlock inputs - you see question, customer_id, chat_history
    • Enter question = "Tell me about the waterproof tents"
    • Enter customer_id = 2
    • Enter chat_history = []
    • Click Execute to see results: You should see a valid response with a list of matching tents from the product catalog with additional details.
  5. ✅ | Congratulations! - Your Chat AI Deployment is working!

Testing

We can think about two levels of testing - manual validation and automated evaluation. The first is interactive, using a single test prompt to validate the prototype as we iterate. The second is code-driven, using a test prompt dataset to assess quality and safety of prototype responses for a diverse set of prompt inputs - and score them for criteria like coherence, fluency, relevance and groundedness based on built-in or custom evaluators.

1️⃣ | Manual Testing (interactive)

The Contoso Chat application is implemented as a FastAPI application that can be deployed to a hosted endpoint in Azure Container Apps. The API implementation is defined in src/api/main.py and currently exposes 2 routes:

  • / - which shows the default "Hello World" message
  • /api/create_request - which is our chat AI endpoint for test prompts

To test locally, we run the FastAPI dev server, then use the Swagger endpoint at the /docs route to test the locally-served endpoint in the same way we tested the deployed version/

  • Change to the root folder of the repository
  • Run fastapi dev ./src/api/main.py - it should launch a dev server
  • Click Open in browser to preview the dev server page in a new tab
    • You should see: "Hello, World" with route at /
  • Add /docs to the end of the path URL in the browser tab
    • You should see: "FASTAPI" page with 2 routes listed
    • Click the POST route then click Try it out to unlock inputs
  • Try a test input
    • Enter question = "Tell me about the waterproof tents"
    • Enter customer_id = 2
    • Enter chat_history = []
    • Click Execute to see results: You should see a valid response with a list of matching tents from the product catalog with additional details.
  1. ✅ | Congratulations! - You successfully tested the app locally
2️⃣ | AI-Assisted Evaluation (code-driven)

Testing a single prompt is good for rapid prototyping and ideation. But once we have our application designed, we want to validate the quality and safety of responses against diverse test prompts. The sample shows you how to do AI-Assisted Evaluation using custom evaluators implemented with Prompty.

  • Visit the src/api/evaluators/ folder
  • Open the evaluate-chat-flow.ipynb notebook - "Select Kernel" to activate
  • Clear inputs and then Run all - starts evaluaton flow with data.jsonl test dataset
  • Once evaluation completes (takes 10+ minutes), you should see
    • results.jsonl = the chat model's responses to test inputs
    • evaluated_results.jsonl = the evaluation model's scoring of the responses
    • tabular results = coherence, fluency, relevance, groundedness scores

Want to get a better understanding of how custom evaluators work? Check out the src/api/evaluators/custom_evals folder and explore the relevant Prompty assets and their template instructions.

The Prompty tooling also has support for built-in tracing for observability. Look for a .runs/ subfolder to be created during the evaluation run, with .tracy files containing the trace data. Click one of them to get a trace-view display in Visual Studio Code to help you drill down or debug the interaction flow. This is a new feature so look for more updates in usage soon.

Deployment

The solution is deployed using the Azure Developer CLI. The azd up command effectively calls azd provision and then azd deploy - allowing you to provision infrastructure and deploy the application with a single command. Subsequent calls to azd up (e.g., ,after making changes to the application) should be faster, re-deploying the application and updating infrastructure provisioning only if required. You can then test the deployed endpoint as described earlier.

Costs

Pricing for services may vary by region and usage and exact costs are hard to determine. You can estimate the cost of this project's architecture with Azure's pricing calculator with these services:

  • Azure OpenAI - Standard tier, GPT-35-turbo and Ada models. See Pricing
  • Azure AI Search - Basic tier, Semantic Ranker enabled. See Pricing
  • Azure Cosmos DB for NoSQL - Serverless, Free Tier. See Pricing
  • Azure Monitor - Serverless, Free Tier. See Pricing
  • Azure Container Apps - Severless, Free Tier. See Pricing

Security Guidelines

This template uses Managed Identity for authentication with key Azure services including Azure OpenAI, Azure AI Search, and Azure Cosmos DB. Applications can use managed identities to obtain Microsoft Entra tokens without having to manage any credentials. This also removes the need for developers to manage these credentials themselves and reduces their complexity.

Additionally, we have added a GitHub Action tool that scans the infrastructure-as-code files and generates a report containing any detected issues. To ensure best practices we recommend anyone creating solutions based on our templates ensure that the Github secret scanning setting is enabled in your repo.

Resources

  1. Prompty Documentation
  2. Azure AI Studio Documentation
  3. Azure AI Templates with Azure Developer CLI

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. Learn more here:

For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Responsible AI Guidelines

This project follows below responsible AI guidelines and best practices, please review them before using this project:

About

This sample has the full End2End process of creating RAG application with Prompty and AI Studio. It includes GPT 3.5 Turbo LLM application code, evaluations, deployment automation with AZD CLI, GitHub actions for evaluation and deployment and intent mapping for multiple LLM task mapping.

Resources

Readme

License

MIT license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 99.1%
  • Other 0.9%