Product Knowledge Agent
Provider SearchPrototype using illustrative dataTC

Feedback & hypothesis validation

Session data for the facilitator. All figures are illustrative demo analytics.

12

Questions asked

8

Answered with strong support

2

Knowledge gaps

1

Source conflicts

83%

Rated helpful

Recent questions

QuestionEvidence supportRatingSource openedGap flagged
How does Provider Search determine which providers appear?Strong source support Helpful YesNo
How frequently is provider data updated?Conflicting sources Helpful Yes Yes
Who owns provider-data updates?Partial source support Not helpful Yes Yes
What changed in the latest release?Strong source support HelpfulNoNo
What should a shopper do if their provider is missing?Strong source support Helpful YesNo
What is the provider onboarding SLA?No supporting evidence foundNot ratedNo Yes

What we are learning

  • Participants who successfully found an answer10 of 12
  • Average perceived time compared with current process4× faster
  • Participants who opened at least one source9 of 12
  • Participants who said citations increased trust11 of 12
  • Questions the current sources could not answer2
  • Participants who would use the tool during normal work8 of 12

Validation notes

  • •Did the participant understand what they could ask?
  • •Was the answer faster to find than their current method?
  • •Was the answer useful enough to act on?
  • •Did the citations increase trust?
  • •Did the participant open a source?
  • •Could the participant tell when evidence was incomplete?
  • •What information did they expect but not find?
  • •Would they use this during their normal work?
  • •What would prevent them from trusting it?