1. Help Center
  2. Tutorials
  3. Office Sandbox Authoring Tutorials

Office Tutorial 11 - Adding Guidance for Learners

This tutorials covers ways to prompt learners to give more guidance in AI driven experiences.

Quick Links

Step 1: Use Notifications as Prompts

Step 2: Use the "Prompt" Field in the Voice Input Resource

Step 3: Add a Screen Dialog Resource to Help the Learner Progress

Step 4: Add a Checklist

In the Headset

Step 1: Use Notifications as Prompts

One way to prompt the learner on what they need to say is to provide a notification. Notifications appear for 5 seconds and then clear on their own. They are the least invasive (but also least noticeable) UI element in a scene. You would normally use notifications if you have used a Speech Recognition Condition ( as opposed to a Voice Input resource). We used Speech Recognition conditions for the first few steps.

In the Conversation script, navigate to the "Stacey Greets User" frame (you may have named it differently - this is the frame in which the character says their opening line):

Add a subframe that opens when the character finishes their line. You could use the event link shortcut on the character action, or you could add the subframe by clicking the '+' at the end of this frame thumb and then dragging the dialog resource icon into the conditions area of the new frame. For variety (and a reminder of this alternative) here is a video on that option. (Notice that we collapse the nested frames below Stacey greets user. This can be useful if you have a long script to keep things organized):

New frame for hint notification

Add a notification to this new frame that says "Greet the employee and invite them to sit down".

Use the same pattern to add another notification to prompt the learner to say "Thanks for coming in". The second notification is added to the "Start Conversation" frame and the third one is added to a new frame that opens after the employee says "No problem, what can I do for you"? :

add two more notifications for hints-1

 

Save and test. You should now see notifications to prompt you on what you need to say.

Step 2: Use the "Prompt" Field in the Voice Input Resource

You have the option of added a prompt if you use a Voice Input resource. 

It is recommended that you use a voice input resource when you want to be able to prompt the user and the Speech Recognition condition when you don't want a prompt. We have used both in the tutorials to demonstrate the options.

Step 3: Add a Screen Dialog Resource to Help the Learner Progress

There may be times when a learner's manner of speaking, turn-of-phrase, or accent is more difficult for the AI to process. In this instance, it can be helpful to add the option to manually select the line they would like to say, instead of having to speak.

This is the strategy:

  • Allow the learner to attempt to drive the conversation with speech.
  • Listen for the "fail" event and open a frame on a timer condition that will offer up the choices after a set amount of time.
  • If the learner manages to continue down one of the other branches (besides fail) we will close the frame with the screen dialog assistance.
  • If the learner is not able to make any progress, a screen will appear that will allow them to select what they would like to say and move forward.

This video shows how to set up the helper screen (and establishes a pattern for adding more helper screens as the scenario progresses):

Set up helper screens pattern

Use a screen dialog resource in this new "helper Screen frame:

set up screen dialog

Now, we need the events fired by the screen dialog resource to trigger the "Specific Feedback" and "Unspecific Feedback" branches:

Recall that we had the Screen Dialog resource fire "Option1" for the unspecific feedback choice and "Option2" for the specific feedback. This video shows you how to add those object events to the branches so that they have an effect:

creating compound condtitions

Finally, if the user speaks and it results in the "fail" event, but then they try again and one of the other branches begins, we no longer want this helper screen to appear. This final video will show you how to use the "Close Frame" resource to stop the helper screen from appearing if we no longer need it:

close helper screen on success

Save and test. Run through the scenario a few times making sure to test the fail ➡️ helper screen flow and the fail ➡️ voice success flow.

Step 4: Add a Checklist

A checklist can be a good guide and reminder for learners. It is more applicable in procedural scenarios, but we will add one here for demonstration purposes.

  1. Create a new script called "Checklist"
  2. Open the new script. And add the UI anchor "Office Wall Left" from the UI Anchors catalog.
  3. Create a new frame and add an "Itemized List" resource. Make sure you add it from the "Guide" section and not the "Inspector" section:

    For an explanation of Guides, Inspectors, and Playable content and the differences between them, read this article.
  4. Attach the Itemized list to the anchor you added to the first frame (A guide always needs an anchor - it can be a positional anchor, like this or an object in the scene). Give the list a title and add three checklist items:
    make a checklist
  5. The three items in the checklist should be "Greet the Employee", "Give Specific Feedback", "Suggest a Constructive Solution":
  6. Launch this "Checklist" script from the Director script.

 

Save and test. You should now see a checklist in the scene. This tutorial guides you on how to use variables to dynamically check the items off as they are completed.

In the Headset

This is how your scenario will look so far:

 

Next Up:  Office Tutorial 12 - Using Variables to Dynamically Update the Checklist