Have you ever wanted to build an AI chatbot that remembers what users said earlier and responds intelligently just like a human? You’re not alone.
In this blog, I’ll show you how to create exactly that using three powerful tools:
- Azure Cosmos DB to store chat history,
- Power Automate to connect everything without writing much code,
- and Azure OpenAI (GPT-4) to generate smart, personalized responses.
Whether you’re working in healthcare, e-commerce, IT support, or just experimenting with AI, this approach will help you build a chatbot that feels truly conversational.
🧰 What You’ll Need
Before we start, make sure you have:
- An Azure subscription
- A Cosmos DB instance (NoSQL API works best)
- Access to Azure OpenAI with a deployed GPT-4 model
- A Power Automate account
- (Optional) A front-end chat interface like Power Apps, a website, or Teams.
🗃️ Step 1: Store Chat History (Any Database Works)
To generate accurate, personalized responses, we first need to store past conversations. In this example, we’re using Azure Cosmos DB, but you can use any database that Power Automate can connect to — like Azure SQL, Dataverse, or even SharePoint.
For Cosmos DB:
- In the Azure Portal, create a new Cosmos DB for NoSQL account.
- Add a database (e.g., ChatDB) and a container (e.g., ChatHistory).
- Each message gets saved as a document. Here’s an example format:
Json:
{
“id”: “msg001”,
“userId”: “user123”,
“timestamp”: “2025-06-24T10:30:00Z”,
“message”: “I want to reschedule my appointment”,
“sender”: “user”
}
You can group messages by userId or even use a sessionId if you want to track multiple conversations.
🔁 Step 2: Create a Power Automate Flow
Now let’s connect everything using Power Automate.
- Go to Power Automate and create a new Instant Cloud Flow (or Automated Flow if you’re triggering it based on an event).
- Add a trigger – this could be from Power Apps, an HTTP request, or even a chatbot message.
- Next, add the Cosmos DB – Get Items action:
- Choose your Cosmos DB account and container.
- Use a query like:
Sql:
SELECT TOP 5 * FROM c WHERE c.userId = ‘user123’ ORDER BY c.timestamp DESC
- This fetches the last few messages from the user so GPT-4 has some context.
- Add a Select action to format the results into a readable string like this:
User: I want to reschedule my appointment.
Bot: Sure, please share a preferred date.
User: Tomorrow morning.
This will be the conversation history you send to OpenAI.
🧠 Step 3: Call Azure OpenAI for a Response
Now it’s time to get that smart reply from GPT-4.
- Add a new HTTP action in Power Automate.
- Fill it in like this:
- Method: POST
- URI:
https://.openai.azure.com/openai/deployments//chat/completions?api-version=2024-03-01-preview
- Headers:
Json:
{
“Content-Type”: “application/json”,
“api-key”: “<your-openai-api-key>”
}
- Body:
Json:
{
“messages”: [
{
“role”: “system”,
“content”: “You are a helpful assistant. Be friendly and respond clearly.”
},
{
“role”: “user”,
“content”: “User: I want to reschedule my appointment.\nBot: Sure, please share a preferred date.\nUser: Tomorrow morning.”
}
],
“temperature”: 0.7,
“max_tokens”: 400
}
- Add a Parse JSON step to read the response and extract just the message part (usually found in choices[0].message.content).
💬 Step 4: Return or Store the Response
You can now use this response however you like:
- Display it in a Power App or web chat
- Send it in an email, SMS, or Microsoft Teams message
- Or store it back in Cosmos DB to keep a full record of the conversation
Example of how the bot reply would look in Cosmos DB:
Json:
{
“userId”: “user123”,
“message”: “Your appointment is rescheduled for tomorrow at 10 AM.”,
“sender”: “bot”,
“timestamp”: “2025-06-24T10:35:00Z”
}
🧠 Bonus Tips
- You can summarize longer chat histories using GPT-4 before sending them to avoid token limits.
- Always sanitize inputs and use role separation in your prompt to prevent prompt injection.
- If you’re working in healthcare or other sensitive fields, make sure you handle data privacy and compliance correctly.
🌍 Where Can You Use This?
This setup isn’t just for one industry — it can be used anywhere:
- Customer support
- HR helpdesks
- Sales chatbots
- Scheduling bots
- IT troubleshooting assistants
Once your chat has memory, the possibilities really open up.
🏁 Conclusion
And there you have it! You just learned how to:
✅ Store chat history in any DB
✅ Use Power Automate to retrieve that history
✅ Call Azure OpenAI to generate a relevant response
✅ Return or store that response for ongoing conversations
This kind of integration brings real conversational intelligence into your apps and it’s easier to build than ever before.
🤝 Need Help Building This?
If you’re still finding it tricky or would like some help setting this up, feel free to reach out we’d be happy to work with you and build it together. Whether it’s customizing it for your business, integrating with your existing systems, or scaling it further we’ve got you covered.