Enable ChatGpt using Azure

Enable ChatGpt using Azure - Enterprise GPT Web App

This guide will help as a step-by-step guide on how to do the following:

Create a GPT-like experience over your own data

Use Azure OpenAI service to access and deploy GPT models

Use Azure Search service for data indexing and retrieval

Enable Integrated Vectorization for cloud based data ingestion

Secure the web application via Microsoft Authentication

Enable user document upload and manage file attachments

Prerequisites An Azure account and subscription - Create one now

Azure account permissions -

Your Azure account must have Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator, User Access Administrator, or Owner. If you don't have subscription-level permissions, you must be granted RBAC for an existing resource group and deploy to that existing group

Your Azure account also needs Microsoft.Resources/deployments/write permissions on the subscription level

Install the required tools:

Azure Developer CLI

Python 3.9, 3.10, or 3.11

Important: Python and the pip package manager must be in the path in Windows for the setup scripts to work.

Important: Ensure you can run python --version from console. On Ubuntu, you might need to run sudo apt install python-is-python3 to link python to python3.

Node.js 18+

Git

Powershell 7+ (pwsh) - For Windows users only.

Important: Ensure you can run pwsh.exe from a PowerShell terminal. If this fails, you likely need to upgrade PowerShell.

Cost Estimation Pricing varies per region and usage. For a personalized quote, refer to the Azure pricing calculator for the resources below:

Azure App Service: Basic Tier with 1 CPU core, 1.75 GB RAM. Pricing per hour. Pricing

Azure OpenAI: Standard tier, GPT and Ada models. Pricing

Azure AI Document Intelligence: SO (Standard) tier using pre-built layout. Pricing

Azure AI Search: Basic tier, 1 replica, free level of semantic search. Pricing per hour. Pricing

Azure Blob Storage: Standard tier with ZRS (Zone-redundant storage). Pricing per storage and read operations. Pricing

Azure Monitor: Pay-as-you-go tier. Costs based on data ingested. Pricing

Getting Started For this guide, we’ll be using the following open-source repository: Azure GPT - RAG

Set up local environment Navigate to the Azure GPT - RAG repository and run the following command to download the codebase: azd init -t azure-search-openai-demo

Before proceeding further, unlink the repository from the current remote origin: git remote remove origin

Link the codebase to a private repository in your own Github organization: Adding locally hosted code to GitHub

Steps (3) and (4) ensure that any future changes you make can be pushed into a private repository

Deploying to Azure App Service This sub-section will enable you to get the chat application up and running with a few commands:

Navigate to azure.yaml file in the root directory

Comment out host: containerapp and uncomment host: appservice

Open a terminal and login to your Azure account: azd auth login

Create a new azd environment to store the deployment parameters: azd env new Example-RG

Enter a name (replace ‘Example-RG’) that will be used for the resource group. This will create a new folder in the .azure folder, and set it as the active environment for any calls to azd going forward.

Set the deployment target to appservice: azd env set DEPLOYMENT_TARGET appservice

Navigate to the data folder in the root directory, and replace the existing files with your own custom data (if any).

The files and folders contained with-in this folder will be ingested, computed as embeddings and stored in a search index. To avoid computing any unncessary documents, remove these files or replace with your own.

Enable Integrated Vectorization. For more details regarding this, please refer to the Data Ingestion section below:

azd env set USE_FEATURE_INT_VECTORIZATION true Provision the resources and deploy the code: azd up

This will provision Azure resources and deploy this sample to those resources.

Open Untitled Project.jpg Once provisioning begins, you will be prompted to choose location for few of the resouces to be created.

You can choose any region based on your preference

Important: For openAiResourceGroupLocation, please refer to the following link. As certain model deployments may not be available in your preferred region choice, therefore choose according to what model you plan to use - https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure%2Cglobal-standard%2Cstandard-chat-completions#global-standard-model-availability

Once the provisioning is complete, it will automatically run the azd deploy command and deployment of services will begin. Please be patient until you see the success message.

Open Screenshot 2024-11-26 at 1.17.33 PM.png Running Local Development Server After a successful execution of the azd up command, now you can run the application locally and start customizing according to your needs.

Authenticate again if needed and run azd auth login

Start the server:

Windows: ./app/start.ps1

Linux/Mac: ./app/start.sh When you run ./start.ps1 or ./start.sh, the backend files will be watched and reloaded automatically. However, this behavior does not apply to the frontend files—they won't be watched or reloaded automatically.

To enable hot reloading for the frontend files, first, wait for the backend to initialize. You’ll know it’s ready when you see the following message in the terminal: Running on http://127.0.0.1:50505

Now open a new terminal and navigate to the frontend directory: cd app/frontend Then run: npm run dev

You should see:

[email protected] dev vite VITE v4.5.1 ready in 957 ms ➜ Local: http://localhost:5173/ ➜ Network: use --host to expose ➜ press h to show help Navigate to the URL shown in the terminal (in this case, http://localhost:5173/). This local server will watch and reload frontend files. All backend requests will be routed to the Python server according to vite.config.ts.

Then, whenever you make changes to frontend files, the changes will be automatically reloaded, without any browser refresh needed.

This is how your current chat application will be looking at this point, both locally and the deployed one:

Open Screenshot 2024-11-26 at 2.14.29 PM.png 6. In the next section, we will focus on customizing this experience and re-deploying your changes.

App Customization App Customization changs are completely optional but we recommend tailoring the frontend and backend according to your needs.

You will find all relevant information regarding the customization inside the original repository: App Customization

Data Ingestion Data ingestion refers to adding your own custom data, that may be specific to your internal organization or domain specific knowlege

Data ingestion can be done using two techniques:

Manual Indexing

Integrated Vectorization

Manual Indexing Manaul indexing process works by adding your data files in the data folder, which are then uploaded into your Azure storage

This data is then ingested into a pipeline, where it is broken down into chunks, embeddings are computed and search indexes are prepared.

For a more in-depth guide into Manual Indexing: Manual Indexing Process

Integrated Vectorization Data ingestion through manual indexing can be cumbersome, requiring you to locally upload new data files and rerun specific scripts—a time-consuming and inefficient process. This is where Integrated Vectorization steps in as a game-changer.

Azure AI Search recently introduced Integrated Vectorization, a cloud-based solution that streamlines the entire data ingestion pipeline. This feature automates critical tasks such as document format cracking, data extraction, chunking, vectorization, and indexing—all powered by Azure technologies, delivering a seamless and efficient approach to managing your data.

If you have skipped enabling this in the deployment to app service section, please follow the guide below:

If you've previously deployed, delete the existing search index. 🗑️

To enable the use of integrated vectorization, run:

azd env set USE_FEATURE_INT_VECTORIZATION true If you've already deployed your app, then you can run just the provision step: azd provision

That will set up necessary RBAC roles and configure the integrated vectorization feature on your search service.

If you haven't deployed your app yet, then you should run the full azd up after configuring all optional features.

Managing Documents (Integrated Vectorization Approach Only) To add additional documents or to remove them from the index, navigate to your data source (Blob storage, by default).

Navigate to the resource group

Find the resource listed as 'Storage account'

Inside your storage account, navigate to the Storage Browser and then to Blob Containers

Upload or remove data files as needed within the content folder

Open Screenshot 2024-11-26 at 4.51.03 PM.png Now navigate to the Search service resource and expand Search managementfrom the side panel

Open up the Indexer and re-run it. Refresh and wait for the Success status

Open Screenshot 2024-11-26 at 5.04.37 PM.png Once successful, you are good to go with the updated blob content. However, if you wish to test your search index

Navigate to Indexesfrom the side panel and query anything from the recently updated data

Open Screenshot 2024-11-26 at 5.05.48 PM.png Additional Features This section will introduce you to a few additional features, that aren’t available/deployed out of the box. Our guide will help you enable the following, to bring you an experience closer to the actual Chat-GPT:

Enable Models such as GPT 4 / GPT-4o / GPT-4o mini

Enable Authentication for controlled access

Enable document level access control and user document upload

Enabling GPT 4 / GPT-4o / GPT-4o mini
By default, your current deployment will be using GPT 3.5 Turbo. However, you can update it with the model of your choice. Execute the following commands inside your terminal:

To set the name of the deployment, run this command with a unique name in your Azure OpenAI account. You can use any deployment name, as long as it's unique in your Azure OpenAI account.

azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT chat4

To set the GPT model name to a gpt-4, gpt-4o, or gpt-4o mini version from the available models, run this command with the appropriate GPT model name.

For GPT-4: azd env set AZURE_OPENAI_CHATGPT_MODEL gpt-4

For GPT-4o: azd env set AZURE_OPENAI_CHATGPT_MODEL gpt-4o

For GPT-4o mini: azd env set AZURE_OPENAI_CHATGPT_MODEL gpt-4o-mini

To set the Azure OpenAI deployment SKU name, run this command with the desired SKU name. azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT_SKU GlobalStandard

To set the Azure OpenAI deployment capacity, run this command with the desired capacity. azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT_CAPACITY 10

To set the Azure OpenAI deployment version from the available versions, run this command with the appropriate version. For GPT-4: azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT_VERSION turbo-2024-04-09

For GPT-4o: azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT_VERSION 2024-05-13

For GPT-4o mini: azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT_VERSION 2024-07-18

To update the deployment with the new parameters, run this command. azd up

Note: You can also manually manage your model deployments via https://oai.azure.com/

Enabling Authentication Two Microsoft Entra applications must be registered in order to enable the optional login. These app registrations are also mandatory to enable document level access control which is required for User Document Upload.

One app is for the client UI. The client UI is implemented as a single page application

The other app is for the API server. The API server uses a confidential client to call the Microsoft Graph API

Follow the guide below to setup and configure the two applications:

Run azd env set AZURE_USE_AUTHENTICATION true to enable the login UI and use App Service authentication by default.

Run .venv/bin/python ./scripts/manageacl.py --acl-action enable_acls(from root dir) to enable access control on your search index.

Run azd env set AZURE_AUTH_TENANT_ID to set the tenant ID associated with authentication.

You can find the tenant ID by navigating to Microsoft Entra ID

Check for the Tenant ID field under Basic information in the Overview tab

Run ./scripts/auth_init.sh and wait for the authentication setup to complete.

You can verify the script’s succession by navigating to Microsoft Entra ID → App Registrations

Ensure you have two new app registrations, namely Azure Search OpenAI Chat Client App and Azure Search OpenAI Chat Server App respectively.

You can complete the authentication setup by launching the azd up command or enable user document upload first by following the next section and then provision the resources by using azd up.

Managing Supported Account Types By default, the app configuration sets the supported account type to Single Tenant.

Single Tenant support allows access to only the accounts in your main tenant’s organizational directory.

You can choose to keep this on default configuration and allow access to people from your organization only, or you can choose to have Multi-Tenant account support (Any Microsoft Entra ID tenant).

To change the supported account types to Multi-Tenant, follow the steps below for both the Client and Server app registrations:

Navigate to the respective app registration

Choose Authentication settings under the Manage options in the side panel

Scroll to the supported account types section and choose Accounts in any organizational directory

Open Screenshot 2024-11-27 at 3.46.20 PM.png Enabling User Document Upload This section covers the user document upload system to allow users to upload their own documents and chat with them.

Important: This feature requires you to first enable authentication.

You can enable the user document upload feature by setting an azd environment variable: azd env set USE_USER_UPLOAD true

If you are enabling this feature on an existing index, you should also update your index to have the new storageUrl field:

.venv/bin/python ./scripts/manageacl.py -v --acl-action enable_acls Finally, run the azd up command.

This guide walks you through the process of building and deploying a complete end-to-end Enterprise GPT application using Azure.