Tips & tricks for using the Power Virtual Agents Boost Conversational Coverage Preview

[Jameslew_MSFT]

Welcome to the Power Virtual Agents boost conversation preview. We encourage you to read the documentation about this new feature here (https://go.microsoft.com/fwlink/?linkid=2226634). In addition, here are some things to be aware of as you try out this new preview capability.

Configuring your bot

• URL depth: The URL you provide for the boost conversation preview represents the scope of content that will be used for retrieving results. The URL you specify can have sub paths, such as "www.microsoft.com/accessibility". The URL can support up to two levels of depth. If you include a top-level URL, such as "www.microsoft.com" all sub paths will be considered. Note that if your site redirects to another site not in the same path as the URL you provided, that content will not be included in results. In addition; the URL "careers.microsoft.com" will search for careers content, however, "microsoft.com" without a prefix will search all domains under "microsoft.com". Boost conversation will not follow http redirects to answer questions.
• Types of content sites: Generally, the boost conversation preview doesn't work as well with forums/social networks in providing answers. Additionally, community content on social networks can often increase the risk of more answers being rejected due to inappropriate/malicious content that is more common on those sites. The URL you specify also should be the content site to be searched, not a search engine itself or other search-based sites, such as weather, stock, news, or job boards. Specifying for example "bing.com" or other search engine in the URL does not provide useful results.
• Moderation dropdown: Use the Bot content moderation dropdown in the AI capabilities page to tune the moderation filters; a higher level of moderation means that the bot’s answers will be more relevant. Setting content moderation to 'Medium' will increase bot creativity and may result in inaccurate responses. Setting content moderation to 'Low' will, in addition, allow queries and responses with offensive content. See more.

Testing your bot

• Escalate to human response: When your PVA bot user asks questions, sometimes conversation boosting will not be able to provide an answer, and will respond with: "I’m sorry, I’m not sure how to help with that. Can you try rephrasing?". Starting with the third question without an answer, PVA responds with a message about escalating to a live agent. This built-in behavior in PVA is designed to ensure your customers can reach assistance when properly configured.
• Under the hood/diagnosing results: The boost conversation preview utilizes Microsoft Bing to retrieve results from your provided URL to retrieve results to questions provided by users. If you want to learn more about the query being used, you can emulate it by passing the question you asked followed by "site: <your URL here>" and see what the few top Bing results are for the question.
• Types of questions that don't work well: The boost conversation preview works well with a large variety of question types. However, there are certain types that won't work well:
• Personal questions
• Questions that require authenticated access to content
• Questions for each there is no content on the specified website.
• Questions that require calculations, comparisons, and/or form submissions to provide answers; for example, terms such as better/latest/cheapest/etc. Currently, this preview feature does not have the capability to do calculated comparisons.
• Questions with offensive, adult, or harmful content. We have also intentionally implemented content moderation to block such content. You can relax this setting by selecting the Content Moderation selection in the AI Capabilities page to 'Low'.
• Single versus multi-turn answers: The boost conversation capability in PVA does not remember context across multiple questions in the conversation. Each boosted conversation post will consider as context only the URL and the latest query, not on the entire conversation. Make sure to provide sufficient context in each query instead of assuming the feature knows about the prior turn.

Giving us feedback

• During this preview phase, we are actively collecting feedback through our support teams, our community channels, and in-product feedback associated with each bot response. The thumbs up, thumbs down UI at the bottom of each boost conversation post will share feedback about your experience with Microsoft that we will use to fine tune the product's capability.
• If you receive irrelevant or inappropriate conversation posts, please give a thumbs down so that we can improve our algorithm. If you receive an empty response when information for the question does exist on your website, we invite you to provide feedback through our support teams or community channels here: General - Power Platform Community (microsoft.com).

We're excited to hear your feedback here in the forums and learn more about your experiences using the conversation boosting capability in PVA!

[hugobernier · Accepted solution]

This is so exciting! If you haven't tried Power Virtual Agents, try it at https://aka.ms/trypva

[Spiff]

This is exiting indeed. Any information on a capability of linking to content on SP365 (internal intranet) as most of employee relevant information in many companies are placed there?  

[Jameslew_MSFT]

There's a link in the UI to sign up for a closed preview for accessing internal docs.  We havent' yet provided a date for when that would be available more broadly.

[aalaam]

great feature, when will this be available on production? it would also be amazing if more than 1 URL can be added not just one

[Jameslew_MSFT]

Coming soon; we don't have a general availability date to share at this time, but we're working on it and will post when we have more info to share.  Support for multiple URL's will also be added to the preview soon as well.

[DT24]

Any way to protect my private data with questions asked or responses received? 

[ichbinchris]

Is there a plan to increase the levels of URL depth? In cases where content is located on enterprise SharePoint sites there is a need to include more than the current two levels (for example https://company123.sharepoint.com/sites/ww/team).

Also, is there a plan to allow websites that end in file extensions like .aspx?

Thanks

[Jameslew_MSFT]

URL Depth should be specific to the public content sites; it's because we utilize bing's "SITE:<url>" parameter in our queries and their max depth for that is two.  You can mitigate that by using a preconfigured Bing custom search instead: https://learn.microsoft.com/en-us/power-virtual-agents/nlu-boost-node#use-a-bing-custom-search-to-se...

PVA doesn't actually index content, we search over existing indexes (like Bing, or the Microsoft graph).  We don't have the ability to directly grovel over a web site.  If your ASPX shows up in Bing or Graph search, it should be searchable in PVA (modulo any authentication requirements, etc. which are not supported.)

[aman_y]

Hi @Jameslew_MSFT, I'm developing a chatbot with copilot and uploaded some PDF documents to leverage the generative answers feature. However, I've noticed that initially, it couldn't provide answers to some questions, but after asking the same question 2-3 times, it is providing response. This has been happening for so many questions. 

What may be the reasons for this? Is there any server issue or what?
Just want to emphasize that the questions asked are very relevant to the information present in documents.