Using Visual Cards in Chat Bot Dialog

Exploring the use of Adaptive Cards in Microsoft's Bot Framework

Introduction

On June 8th, Nicholas Cipollina wrote a great blog on Getting Started With Bot Framework. In our tutorial today we are looking to expand on his Froyo Bot, and incorporate a cool feature that was recently added to the Bot Framework. Since we will be using the bot that was created in Nick's tutorial, it would be beneficial to work through the Getting Started blog first and then come back to this post. In our tutorial we will expand on the Froyo Bot to add Adaptive Cards. Adaptive Cards will give the bot a different way for the users to interact with it, and introduce you to a different method of displaying information to a user that is chatting with your bot.

Adaptive Cards are a newer feature to the Bot Framework, and while still in preview mode, Adaptive Cards are a valuable tool for developers to use inside and outside of chatbots. Adaptive Cards can be a great option for displaying meaningful information to users where text might not be the best option.

If you would like to read about the Adaptive Cards framework, you can do so at adaptivecards.io. I recommend checking out some of the samples, as well as the visualizer, so you can understand their capability and how we will be formatting our cards.

Getting Started

For our tutorial today, we will be using the Microsoft Visual Studio IDE on Windows. We will also be using the following NuGet package in addition to what was used in the Froyo Bot tutorial:

  • Microsoft.AdaptiveCards v0.5.1

Once again, we will be building off of Nick's previous work in the Getting Started With Bot Framework blog, so I highly recommend doing that tutorial first.

Changing The Design and Preparing For Cards

To start our project, let's open up the Froyo bot that we made before, and then change the Conversation.sendAsync() method back to calling our Root Dialog. In the image below I commented the old code out and placed in the new code.

Basic Activity

Now that we are calling our root dialog again, we need to add some substance to that dialog so that we can prepare to add our cards into the bot. Let's start by adding a menu where users can either choose to build their own Froyo or pick from a selection of customer favorites!

To keep things simple for the user, we will have our menu as a prompt. Prompts work similar to the Froyo ordering form. To add a prompt, enter the code below in the MessageReceived method.

Sample MessageReceived method

This prompt will simply ask the user to pick two options, and give them 3 attempts to do so before canceling the Dialog with them. If you would like to read more about Prompts in Dialogs with the Bot Framework, you can find additional documentation here.

Now, let's finish up our menu and move on to our Customer Favorites Selection. Change the code below for your MenuChoiceReceivedAsync method so that we can handle the choice from the user.

Sample MenuChoiceReceivedAsync Method

Change your FroyoOrder model to include the BuildFormDialog method that we need to call the form, as well as the new welcome message so that we aren't repetitive.

Sample Build Form Dialog

We have added a few things into our Customer Favorite case in the "if" statement, but we will handle those in the next section when we start to talk about Adaptive Cards.

Adding in Our Customer Favorite Cards

To start working with Adaptive Cards, the first thing we want to do is create our Adaptive Card Model, FavoriteModel, that we will use to display the Customer Favorites to our user.

Sample Favorite Model

We will create our Adaptive Card as an attachment, because we want to append it onto our prompt message in our Customer Favorite case in the if statement. We will also take the variables name, size, flavor, and toppings so that we can make each card unique.

Before we design our card in the model, we need to use the visualizer first to determine what we want our card to look like. The visualizer is a great tool to prototype and define how you want your card to look before you start to code.

Sample Card

I settled on this design, which shows the name of our Froyo favorite, the size, flavor, and toppings of the Froyo, and a button at the bottom to order the Froyo.

Once we have a design set, we can implement that same design in our Adaptive Card within the bot. While I always recommend typing out code instead of copying it, I have provided the code for the Adaptive Card below, because it is a pain to type out.

// Creating the Adaptive Card we will use 

AdaptiveCard FavoriteCard = new AdaptiveCard()
{
 // Defining the Body contents of the card 
 Body = new List()
 {
 new Container()
{ Items = new List() { new TextBlock() { Text = name, Weight = TextWeight.Bolder, Size = TextSize.Medium }, new ColumnSet() { Columns = new List() { new Column() { Items = new List() { new Image() { Url = "127.0.0.1", Size = ImageSize.Large } } }, new Column() { Items = new List() { new FactSet() { Facts = new List() { new AdaptiveCards.Fact() { Title = "Size", Value = size }, new AdaptiveCards.Fact() { Title = "Flavor", Value = flavor, }, new AdaptiveCards.Fact() { Title = "Toppings", Value = toppingsString } } } } } } } } } }, // Defining the actions (buttons) our card will have, as well as their functions Actions = new List() { new SubmitAction() { Title = "Order", // returning our StringData as JSON to the bot DataJson = StringData } } };

We also want to add a few variables for our card to use. These make formatting easier in the bot and helps us later with returning some values to the bot that we can parse through.

Bot Variables

If you don't feel like typing it out, here is the code:


// Setting a string of toppings for JSON
string toppingsStringforJSON = string.Join("\",\" ", toppings.ToArray());

// Setting a string of toppings to be used in the cards
string toppingsString = string.Join(", ", toppings.ToArray());

// Compiling all of the variables provided for JSON

string StringData = "{ \"name\": \"" + name +

"\", \"size\":\"" + size +

"\", \"flavor\":\"" + flavor +

"\", \"toppings\":[\"" + toppingsStringforJSON +

"\"]}";

This creates three strings. We will use the first string, toppingsStringforJSON, for our JSON formatting in the StringData string, and the toppingsString will be used for listing all of the selected toppings in the card view.

Below our actions, we want to define the attachment we will be sending back and attach our card to that attachment. Finally, we want to return that attachment to the dialog calling our method.

Returning Card Data

Now that we have a fully written card, as well as information to feed it, let's go ahead and run our bot and test it in the emulator. Once you have it running, select the favorites button and our Froyo favorite that we have added! You should notice that the bot takes your order, as we can see in the data on the right side, and sends a message back to you thanking you for your order!

Viewing our Data

Now, having one special to order is pretty cool, but what if we had more than one? What about a whole carousel of favorites!?

Let's modify our code in the Root Dialog to handle more than one attachment, as well as add those attachments to our message. I included some ideas for extra Froyo favorites below. Feel free to use them or get creative with your own!

Adding More Cards

prompt.Attachments.Add(FavoriteModel.CustomerFavoriteCard("Smore's Delight",
"Medium",
"Chocolate",
new List { "Graham Cracker Crumbs", "Fudge", "Cookie Dough", "KitKat", "Chocolate Chips" }));
prompt.Attachments.Add(FavoriteModel.CustomerFavoriteCard("Strawberry & Banana Smoothie",
"Medium",
"Strawberry",
new List { "Strawberries", "Banananas"}));
prompt.Attachments.Add(FavoriteModel.CustomerFavoriteCard("Caramel Coffee Addict",
"Large",
"Espresso",
new List { "Caramel", "Heathbar"}));

Let's fire up the bot and test it! You should get a Carousel as seen in the image below.

Testing our Final Bot

If you got it, great! If not, check your error messages and see where you might have gone wrong.

From here, you can expand on the bot to add images, perhaps send the user a receipt for their order!

Conclusion and Thoughts

Cards can be a very valuable resource, but they are not the solution to displaying all data. Within my own work, I have found cards to be valuable, but also inappropriate for some situations. It's important to note how much information you need to give to the user, and how to keep the feel of a conversation going without breaking the flow. An incorrectly placed card response could kill the flow in a conversation, or leave a user more confused than when they started chatting.

As we saw above, cards can be a very valuable tool to display information to users in a format that can take advantage of visuals and custom made displays. Cards within chatbots can provide the user with a more visual menu, a form to fill out, or even formatted data in response to a query that would just be too messy to return in a text form. When used correctly, cards will be an invaluable addition to your chat bot, make the conversation flow easier and keep the dialog from the bot formatted well along the way.

Notes and Further Reading: